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ABSTRACT 
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and  MOA.  A  new  structural  framework  is  developed  using  Marine  Corps  doctrine  in  order 
to  categorize  MOEs  with  potential  applicability  to  Marine  Corps  C'^I  system  analyses.  The 
framework  is  used  to  compare  and  contrast  identified  MOEs,  including  performance 
ranking  based  on  expert  opinion.  Conclusions  discuss  the  utility  of  different  evaluation 
techniques  and  estimated  performance  of  identified  MOEs.  Additionally,  suggestions  for 
higher  resolution  sub-categories  are  presented. 
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EXECUTIVE  SUMMARY 


The  ongoing  technical  revolution  spawned  by  increasingly  powerful  and  inexpensive 
information  technology  has  given  rise  to  what  has  been  described  as  a  "revolution  in 
military  affairs."  This  revolution,  coupled  with  jointness  issues  and  decreasing  service 
budgets  has  provided  an  impetus  to  use  emerging  technology  as  a  force  multiplier  on 
future  battlefields. 

The  military  is  seeking  to  exploit  technology  in  order  to  provide  integrated, 
interoperable  systems  that  meet  the  command  and  control  requirements  of  services  with 
inherently  different  missions.  Command  functions,  and  the  flow  of  information  on  the 
battlefield  falls  within  the  realm  of  Command,  Control,  Communications,  Computers,  and 
Intelligence  (C"*!).  Increasing  demand  for  imagery,  access  to  national  intelligence 
databases,  sensor  management,  and  wide  area  networks  are  just  a  few  of  the  challenges 
facing  the  services  in  general,  and  the  Marine  Corps  specifically.  Past  shortcomings  in  C'*! 
system  procurement,  however,  coupled  with  the  high  cost  of  emerging  technologies,  have 
made  upgrades  to  current  C'‘I  systems  and  new  C'*!  acquisitions  the  subject  of  intense 
analysis. 

This  paper  presents  the  results  of  an  extensive  literature  review  and  survey  analysis 
conducted  to  identify,  compare,  and  contrast  various  C'*!  evaluation  techniques,  and 
existing  C**!  measures  of  effectiveness  (MOEs).  Existing  MOEs  from  past  military 
analyses  and  academia  are  identified,  as  well  as  potential  methodologies  for  developing 
new  MOEs.  A  structural  framework  is  developed  using  Marine  Corps  doctrine  in  order  to 
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categorize  MOEs  with  potential  applicability  to  Marine  Corps  C"*!  system  analyses.  The 
framework  is  used  to  compare  and  contrast  identified  MOEs,  including  performance 
ranking  based  on  expert  opinion. 

A  multitude  of  C”*!  evaluation  methodologies  exist,  each  approaching  the 
problem  of  comparative  analysis  in  a  slightly  different  manner.  Available  techniques 
include  th^-  Cost  and  Operational  Effectiveness  Analysis  (COEA),  the  Modular  Command 
and  Control  Evaluation  System  (MCES),  the  Mission  Oriented  Approach  (MOA),  the 
Headquarters  Effectiveness  Analysis  Tool  (HEAT),  evolutionary  upgrade  paths,  and 
functional  decomposition  methods. 

The  COEA  is  well  established  as  the  tool  of  choice  for  Marine  Corps  C"*!  system 
comparative  analyses  during  the  concept  exploration  phase  of  the  acquisition  process. 
The  COEA  framework,  however,  is  broad  enough  to  accept  input  from  other,  potentially 
more  precise,  evaluation  techniques.  The  strength  of  the  MCES  lies  in  its  initial  focus  on 
problem  definition  and  the  environment,  rather  than  a  premature  focus  on  a  quantitative 
model.  MCES  also  enjoys  a  commonality  of  terms  and  structure  which  simplify  its 
application.  The  MOA  focuses  on  obtaining  required  input  from  political  and  military 
decisionmakers  prior  to  defining  an  analytical  question.  HEAT  has  been  successfully 
applied  to  military  O'*!  systems,  and  represents  a  systematic  methodology  using 
pre-defined  measures.  Evolutionary  upgrade  path  concepts  help  provide  greater 
resolution  when  comparing  alternate  systems  with  similar  capabilities.  Functional 
decompositions  use  varied  approaches  which  couch  an  analysis  in  terms  of  required 


system  capabilities.  Although  any  hybrid  combination  of  methodologies  would  only  be 
developed  in  the  context  of  the  analysis  at  hand,  one  such  combination  that  suggests  itself 
might  be  a  COEA  which  employs  the  first  four  modules  of  MCES  for  problem  definition, 
and  evolutionary  upgrade  paths  for  distinguishing  the  best  of  a  subset  of  "good"  systems. 

The  identification  of  realistic  MOEs  is  a  common  thread  which  binds  the  various 
C”*!  evaluation  methodologies.  Unique  MOEs  may  be  required  for  a  specific  analysis,  and 
can  be  generated  using  MCES,  MOA,  evolutionary  upgrade  paths,  or  functional 
decompositions.  Importantly,  many  useful  MOEs  already  exist  from  past  analyses.  O'*! 
literature,  and  academia.  The  ACCES,  for  instance,  uses  a  pre-defined  set  of  measures 
which  remain  constant  from  analysis  to  analysis.  The  proposed  framework  for 
categorizing  MOEs  in  Chapter  III,  based  on  relevant  doctrine,  provides  a  tool  for 
identifying  and  maintaining  those  MOEs  with  broad  applicability  to  Marine  Corps  C"! 
system  evaluation.  The  framework  can  change  dynamically  with  emerging  doctrine,  and 
can  also  incorporate  new  MOEs  developed  by  the  application  of  MCES  or  other  MOE 
generating  methodologies. 

Importantly,  the  framework  provides  a  starting  point  for  increased  resolution  in 
categorizing  C**!  MOEs  for  use  in  Marine  Corps  related  analysis.  Potential  sub-categories 
that  may  provide  greater  detail  include  functionality  (i.e..  Is  a  component  primarily  for  C^ 
communications,  intelligence,  or  interoperability?),  and  compatibility  (i.e..  Is  the  system 
employable  inside  existing  vehicle  spaces  and  command  post  configurations?). 
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1.  INTRODUCTION 


A.  BACKGROUND 

The  technological  revolution  spawned  by  the  increasingly  powerful  and  inexpensive 
micro-chip  has  given  rise  to  what  many  call  the  "information  age."  Just  as  personal 
computers  have  transformed  the  commercial  and  private  sectors,  the  marriage  of 
computers  and  increasingly  capable  transmission  systems,  including  satellites  and 
microwave,  have  had  a  dramatic  impact  on  the  American  military.  Coupled  with  sweeping 
changes  in  technological  capabilities  are  transformations  caused  by  the  cessation  of  the 
Cold  War. 

Faced  with  ever  decreasing  budgets  and  end  strength,  the  Army,  Navy  and  Air  Force 
are  redefining  service  missions  that  once  focused  extensively  on  the  former  Soviet  Union; 
the  Marine  Corps  has  enjoyed  greater  stability,  since  that  service  had  only  a  limited  role  in 
the  plan  to  defend  Europe  against  the  Warsaw  Pact.  Similarly,  the  issue  of  jointness  has 
become  dominant,  meaning  all  programs,  especially  those  dealing  with  information  flow, 
are  measured  by  their  potential  impact  on  the  battlefield  interaction  of  different  services. 

Traditionally,  the  flow  of  information  on  the  battlefield  has  been  called  "command  and 
control,"  or  Cl  For  the  purposes  of  this  paper,  command  is  a  function  of  authority  and 
responsibility  assigned  to  an  individual,  while  control  is  the  means  by  which  the 
commander  exercises  command.  Taken  together  C^  may  be  defined  as  the  exercise  of 
authority  over  designated  forces  in  the  accomplishment  of  a  mission.  C^  has  been 
described  as  the  most  important  activity  in  war,  although  C^  "will  not  make  a  ^ngle  attack 
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against  an  enemy  force,  destroy  a  single  target,  or  effect  a  single  emergency  resupply, 
without  C^,  campaigns,  battles  and  organized  engagements  are  impossible."  [Ref  1],  As 
will  be  defined  later,  several  terms  exist  to  describe  the  equipment  and  procedures  used  for 
command  and  control,  which  will  initially  be  described  as  command,  control, 
communications,  computers,  and  intelligence  (C^'l)  systems.  The  Department  of  Defense 
currently  has  in  use,  or  under  development,  a  multitude  of  C^I  systems,  all  striving  to  meet 
the  requirements  implicit  in  the  definition  of  command  and  control. 

The  military  is  seeking  to  exploit  rapid  advances  in  CT  technology,  while 
simultaneously  working  for  integrated,  interoperable  systems  that  meet  the  command  and 
control  requirements  of  inherently  different  service  missions.  DoD  Direction,  as  stated  in 
the  Corporate  Information  Management  initiative  and  "C4I  for  the  Warrior"  [Ref  2]  has 
outlined  the  need  for  more  use  of  Commercial-off-the-shelf  (COTS)  items  and  looser 
business-driven  standards  [Ref.  3].  Increasing  demand  for  imagery,  access  to  national 
intelligence  databases,  sensor  management,  and  wide  area  networks  are  just  a  few  of  the 
challenges  facing  the  services  in  general,  and  the  Marine  Corps  specifically. 

In  the  Marine  Corps,  current  systems  vary  greatly  in  age  and  capability.  New  systems 
often  represent  quantum  leaps  in  capability  due  to  the  age  of  the  systems  they  replace, 
although  the  "new"  system  often  represents  less  then  cutting  edge  technology  due  to  the 
length  of  the  acquisition  process.  The  new  Single  Channel  Ground-Air  Radio  System 
(SINCGARS),  which  is  1980's  technology,  is  currently  being  fielded  to  Fleet  Marine  Force 
units  to  replace  a  family  of  radios  representing  1950's  technology,  which  was  itself  fielded 
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in  the  1960's.  Additionally,  most  of  the  Cl  systems  currently  in  use  were  constructed  as 
stand-alone  systems  designed  to  meet  requirements  at  specific  echelons  of  command  or 
within  certain  warfighting  areas  (e.g.,  air  defense).  Such  design  has  hindered  attempts  to 
modernize  existing  C^I  systems,  since  seamless  connectivity  between  echelons,  warfighting 
areas,  and  services  is  necessary.  For  example,  the  digital  "backbone"  of  today's  Marine 
Corps  employs  approximately  nine  different  pieces  of  communications  equipment,  and 
extends  downward  only  to  the  regimental  level  [Ref  4].  Even  those  newer  equipments 
fielded  for  the  joint  arena  are  not  immune  to  connectivity  problems;  the  Tri-Service 
Tactical  (Tri-Tac)  Unit  Level  Circuit  Switch  (ULCS),  an  Air  Force  program  jointly 
endorsed  by  the  Army  and  Marines,  has  been  purchased  in  quantity  only  by  the  Marine 
Corps,  causing  significant  interoperability  problems  between  Marine,  Army  and  Air  Force 
field  headquarters  [Ref  5]. 

The  effort  to  manage  the  controlled  flow  of  huge  quantities  of  information  over  vast 
distances,  to  increasingly  smaller  units,  has  resulted  in  tremendous  challenges  to  the 
Marine  Corps  research  and  development  (R&D)  community.  Due  to  the  rapid  changes  in 
technology,  and  the  large  dollar  amounts  involved,  upgrades  to  current  systems  and  new 
system  acquisitions  are  the  focus  of  intense  analysis.  While  analysis  techniques  vary,  the 
requirement  to  identify  appropriate  measures  of  effectiveness  (MOEs)  is  fundamental  to 
each.  An  effective  MOE  provides  the  necessary  fi'amework  to  move  an  analysis  from  a 
qualitative  to  a  quantitative  assessment  of  the  effectiveness  of  a  C*I  system.  Although  the 
definition  will  be  expanded  further,  an  MOE  will  initially  be  defined  as  a  quantity  that 
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results  from  the  comparison  of  various  measures  of  a  systems  performance  to  the  mission 
requirements  [Ref  6],  If  MOEs  are  found  which  can  be  applied  across  systems, 
comparisons  between  systems  may  be  facilitated.  Ultimately,  the  difficulties  encountered 
by  some  €“1  systems  during  concept  development,  acquisition,  or  fielding  may  be  the 
direct  result  of  less  than  robust  MOEs. 

B.  PROBLEM  STATEMENT 

A  wide  variety  of  methodologies  have  been  developed  to  evaluate  Cl  systems, 
including  the  modular  command  and  control  structure  (MCES),  the  mission  oriented 
approach  (MOA),  the  cost  and  operational  effectiveness  analysis  (COEA),  and 
evolutionary  upgrade  paths.  A  common  thread  which  binds  these  and  other  approaches  is 
the  fundamental  requirement  to  identify  robust,  realistic,  MOEs.  This  thesis  identifies, 
documents,  compares,  and  contrasts  techniques  for  developing  Cl  MOEs,  and  identifies 
and  examines  existing  MOEs,  especially  those  with  recurring  use,  that  may  be  broadly 
applicable  to  existing  Marine  Corps  programs.  The  usefulness,  definition,  and 
rationale  for  each  of  the  MOEs  are  presented,  as  well  as  the  procedures  for  their 
calculation. 

C.  APPROACH  AND  SCOPE 

A  comprehensive  literature  search  has  been  conducted  in  order  to  fully  document 
existing  methodologies  for  comparing  Cl  systems,  focusing  on  techniques  for  developing 
C^I  MOEs.  Frequently  used  MOEs  are  identified,  as  well  as  MOEs  which  may  have 
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applicability  across  different  C"*!  systems.  The  nature  of  the  MOEs  identified,  as  well  as 
the  procedures  for  their  calculation,  and  the  feasibility  of  the  measure  with  regard  to  cost 
and  time,  are  compared  with  other  measures  to  determine  contrasts.  The  study  of  MOEs 
focuses  on  those  that  may  prove  useful  to  the  Marine  Corps. 

D.  OVERVIEW  OF  REMAINING  CHAPTERS 

1.  Chapter  n.  Literature  Review 

This  chapter  presents  a  current  overview  of  the  large  body  of  literature  dealing  with 
C'*!  effectiveness  measures.  Key  terms  are  defined,  and  a  standard  set  of  terminology  is 
established.  Each  of  the  techniques  for  evaluating  O'*!  systems  is  looked  at  in  overview  in 
order  to  provide  a  framework  of  how  MOEs,  once  established,  are  employed.  The 
techniques  for  MOE  development  inherent  in  each  evaluation  methodology  are  then 
presented,  followed  by  a  discussion  of  the  general  nature  of  the  MOEs  resulting  fi'om 
each. 

2.  Chapter  HI.  Analysis  of  Measures 

This  chapter  establishes  a  structural  framework  in  which  to  place  Marine  Corps  C^ 
MOEs.  The  framework  is  derived  from  Fleet  Marine  Force  Manual  (FMFM)  3 
"Command  and  Control,"  and  is  inclusive  of  current  Marine  Corps  C^  doctrine.  Next, 
pertinent  C^  MOEs  from  the  literature  and  past  analyses  are  positioned  inside  the  doctrinal 
framework,  which  consists  of  five  category  "bins,"  in  order  to  facilitate  analysis  of  the 
measures.  Expert  opinion  is  garnered  to  rank  the  MOEs  on  an  interval  scale  within 
categories,  according  to  the  criteria  of  performance.  Finally,  in  conjunction  with 
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Appendix  A,  a  discussion  of  MOEs  by  category  is  presented,  defining  their  individual 
characteristics,  completeness,  scalability,  and  underlying  measures.  Utilizing  the 
underlying  measures,  the  procedures  for  collecting  measurable  data  and  computational 
requirements  are  discussed. 

3.  Chapter  IV.  Conclusions 

This  chapter  presents  a  summary  of  C”*!  evaluation  methodologies  and  techniques  for 
generating  Cl  MOEs.  The  si  .ngths  of  each  evaluation  method  are  reviewed,  as  well  as 
potential  hybrid  combinations  of  methodologies.  Finally,  an  overview  of  the  doctrinal 
categorical  fi-amework  is  presented,  along  with  suggestions  for  resolution  enhancements  to 
the  existing  categories. 
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n.  LITERATURE  REVIEW 


A.  DEFINITIONS 

1.  Why  Cl? 

The  continuing  rapid  rate  of  change  in  electronics  and  communications  technologies 
has  resulted  in  a  serious  lack  of  uniformity  regarding  terminology  associated  with 
command  and  control.  Different  military  services,  and  indeed  branches  within  services, 
lack  agreement  in  the  definitional  framework  of  [Ref  7],  In  the  years  following  World 
War  II,  the  familiar  term  command  and  control  was  expanded  to  include  communications, 
becoming  C^  Intelligence  was  added  to  create  the  Cl  definition,  which  was  standard  for 
almost  thirty  years  [Ref  8],  This  paper  will  use  the  term  command,  control, 
communications,  computers  and  intelligence  (Cl),  which  reflects  current  usage  in  the 
Marine  Corps  (e.g..  Assistant  Chief  of  Staff,  CT),  and  which  participants  of  the  1993 
Command  and  Control  Concept  Development  Game  agreed  should  become  standard  for 
the  naval  service,  in  order  "to  avoid  unnecessary  confusion  among  ourselves  and  with 
other  services  and  remove  any  differences  between  the  naval  service's  approach  to  C'*!  and 
the  Joint  Staffs."  [Ref  8] 

The  breadth  of  the  C^  domain,  however,  continues  to  give  rise  to  a  multitude  of 
similar  terms.  The  increased  reliance  on  large  intelligence  databases  and  service 
interoperability  requirements  has  given  rise  to  "command,  control,  communications, 
computers  intelligence  and  interoperability  (C**!^,"  and  the  electronic  warfare  community 
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frequently  adds  EW  to  the  above  terms,  as  illustrated  by  the  1992  Military  Operations 
Research  Society  (MORS)  C^IEW  Measures  of  Effectiveness  Conference. 

A  starting  point  for  the  definition  of  command  and  control  was  defined  in  Chapter  I 
as  the  exercise  of  authority  over  designated  forces  in  the  accomplishment  of  a  mission. 
More  specifically,  Joint  Chiefs  of  Staff  Publication  1  (JCS  Pub  1)  defines  Command  and 
Control  as 

The  exercise  of  authority  and  direction  by  a  properly  designated 
commander  over  assigned  forces  in  the  accomplishment  of  his  mission. 
Command  and  control  functions  are  performed  through  an  arrangement  of 
personnel,  equipment,  communications,  facilities,  and  procedures  which  are 
employed  by  a  commander  in  planning,  directing  coordinating,  and  controlling 
forces  and  operations  in  the  accomplishment  of  this  mission. 

Thus  command  and  control  is  by  definition  composed  of  three  major  elements: 
personnel,  equipment,  and  procedures.  In  keeping  with  the  intent  of  past  MORS,  NPS, 
and  MITRE  workshops  dealing  with  the  subject  of  C^  analysis,  C^  should  be  taken  to  mean 
as  broad  a  concept  as  possible,  avoiding  any  distinction  between,  for  example,  "command 
and  control"  and  "combat  direction."  [Ref  7]  Since  this  paper  deals  with  the  identification 
of  MOEs  to  aid  in  the  quantifiable  analysis  of  C^I  "systems,"  the  concept  of  a  O'*!  system 
must  be  defined. 

Again  turning  to  JCS  Pub  1,  a  system  is  defined  as  "the  facilities,  equipment, 
communications,  procedures,  and  personnel  essential  to  a  commander  for  planning, 
directing,  and  controlling  operations."  The  Marine  Corps  has  described  a  Cl  system  as 
"the  integration  of  communications,  computers  and  intelligence  technologies  and 
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procedures  into  a  functional,  cohesive  system  designed  to  support  the  commander  .  .  .  C"*! 
is  distinct  from  in  that  C*l  is  the  enabler  for  C^."  [Ref  8]  Although  emerging  Marine 
Corps  doctrine  seems  to  define  the  components  of  a  system  as  people,  information,  and 
support  structure,  for  purposes  of  this  paper  a  O'*!  system  will  be  viewed  as  having  as 
its  three  components;  physical  entities,  structures,  and  processes  [Ref  1],  Again  using 
definitions  from  MORS  workshops  and  the  MCES  system,  a  C*l  system  incorporates  the 
three  elements  of  the  JCS  Pub  1  definition  in  the  following  manner. 

a.  Physical  Entities 

Physical  entities  refers  to  the  equipment,  software,  facilities,  and  manpower  of  a 
O'*!  system.  Manpower,  or  people,  "drive  the  command  and  control  system  -  they  make 
things  happen  -  and  the  rest  of  the  system  exists  only  to  serve  them."  [Ref  1]  In  other 
words,  humans  are  integral  components  of  the  Cl  system,  not  merely  users  [Ref  IJ. 
Facilities  refer  to  "the  command  posts  and  other  spaces  where  command  and  control  is 
sometimes  performed,"  while  equipment  includes  "the  communications  and  other  gear 
which  support  command  and  control,  from  pyrotechnics  to  computers."  [Ref  1] 

b.  Structures 

Structures  of  a  C'*!  system  identify  the  arrangement  and  interrelationships  of 
physical  entities,  procedures,  protocols,  concepts  of  operation,  and  information  patterns 
[Ref  9].  Such  arrangements  can  be  both  spatial  and  temporal.  Procedures  include  the 
"techniques  by  which  we  perform  various  command  and  control  functions,"  and  th^  Jq»ply 
only  to  "rote  or  mechanical  tasks,  not  to  acts  of  judgement."  [Ref  1]  Organization 
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"provides  those  groups  of  people  through  which  the  commander  exercises  command  .  .  . 
and  establishes  the  channels  by  which  information  flows."  [Ref  1] 
c.  Processes 

A  process  is  a  description  of  what  the  system  is  actually  doing,  and  reflects  the 
functions  carried  out  by  the  system  [Ref  1],  These  activities  "include  -  but  are  not  limited 
to  -  gathering  and  analyzing  information,  making  decisions,  organizing  resources, 
planning,  communicating  instructions  and  other  information,  coordinating,  monitoring 
results,  and  supervising  execution."  [Ref  1]  Note  that  processes  are  not  the  same  as 
procedures.  Both  processes  and  functions  are  defined  in  detail  in  the  Marine  Corps 
Mission  Area  Analysis  of  Mission  Area  11  -  Command  and  Control. 

2.  Measures 

Within  the  realm  of  C'*!  system  evaluation,  the  set  of  terms  that  has  been  developed 
by  the  analytic  community  seems  to  enjoy  a  more  uniform  acceptance  than  the  previous 
terms  dealing  with  the  broader  descriptions  of  C^.  Measures  of  C'*!  systems  cover  a 
spectrum  of  specific  system  aspects.  The  phrase  "measures  of  merit"  (MOM)  has  been 
used  to  describe  the  overall  collection  of  measures,  which  will  be  categorized  according  to 
four  groups;  measures  of  force  effectiveness  (MOFEs),  measures  of  effectiveness 
(MOEs),  measures  of  performance  (MOPs),  and  physical  (sometimes  called  dimensional) 
parameters  [Ref  9].  A  hierarchical  relationship  exists  among  MOMs,  as  portrayed  in 
Figure  1 . 
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Figure  1.  Hierarchical  Relationship  of  Measures  of  Merit.  From  Ref  [9], 


MOMs,  as  listed,  are  categorized  by  their  relationship  to  the  boundaries  of  a  C*I 
system.  In  turn,  the  boundary  is  defined  by  the  scope  of  the  analysis  being  conducted,  and 
represents  the  delineation  between  the  system  being  studied  and  the  environmait. 
Therefore  the  definitions  of  the  four  categories  of  measures,  while  rigorous,  can  only  be 
applied  in  their  relationship  to  the  system  boundary.  Figure  2  depicts  the  relationship 
between  different  boundaries  and  the  environment,  in  what  is  called  the  "onion  skin* 
diagram.  Thus  a  thorough  understanding  of  the  boundaries  for  any  system  must  be 
accomplished  within  the  context  of  the  analysis  at  hand.  Depending  on  the  resolution  of 
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Besides  category  and  system  reference  (or  boundary)  attributes,  all  MOMs  have  the 
additional  attributes  of  names,  function  reference  (or  purpose),  units  of  measure,  value 
measured,  and  threshold  value  (or  goal)  [Ref  9] 

0.  Physical  Parameters 

A  physical  parameter  describes  the  tangible,  physical  aspects  of  a  component  or 
system.  Its  value  determines  system  behavior  and  the  structure  under  question  even  when 
at  rest  [Ref  7],  Examples  of  physical  parameters  include  size,  weight,  capacity,  and 
number  of  pixels. 

b.  Measures  of  Performance 

An  MOP  is  closely  related  to  a  physical  parameter,  in  that  it  represents  a 
property  or  characteristic  inherent  in  the  physical  entity  [Ref  9].  Stated  in  another  way,  it 
measures  attributes  of  systems  behavior,  or  "performance  is  a  measure  of  the  ability  to 
take  action  in  accordance  with  requirements  in  the  execution  of  an  action."  [Ref  6] 
Examples  include  baud  rate,  throughput,  and  frequency  range.  MOPs  are  internal  to  the 
system  being  analyzed,  and  are  scenario  independent  (Ref  9]. 

MOPs  have  also  been  defined  as  non-probablistic  measures  of  performance, 
where  "the  MOP  class  provides  for  the  collection  of  metrics  . .  .  that  are  not  probabilities 
of  successfiil  outcomes  of  functions."  [Ref  10]  Thus  MOPs  are  the  "consequence"  of 
specific  configurations  of  physical  elements.  The  probabilistic  nature  of  this  definition  of 
MOPs  also  leads  to  an  alternate  definition  of  MOEs. 
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c.  Measures  of  Effectiveness 

An  MOE  is  a  measure  of  how  the  C'*!  system  performs  its  functions,  or  affects 
other  entities,  within  an  operational  environment  [Ref  9],  C'‘I  MOE  are  measured 
relative  to  some  standard,  which  is  often  simply  the  implicit  baseline  of  "how  would  a 
perfect  Cl  system  perform."  [Ref  9]  Additionally,  C^I  MOE  are  scenario  dependent. 
Some  examples  include  probability  of  detection  and  target  detection;  Chapter  III  will 
provide  many  more  examples  in  the  form  of  O'*!  MOEs  with  potential  applicability  to 
Marine  Corps  systems. 

By  couching  the  definition  in  probabilistic  terms,  Girard  [Ref  10]  and  Elele  [Ref 
6]  have  developed  more  mathematically  rigorous  definitions  for  MOEs.  In  Girard's  terms, 
an  MOE  is  the  probability  of  the  successful  accomplishment  of  a  function,  where  all 
probabilities  are  conditional,  and  are  derived  from  MOPs  and  lower  level  (or  prior)  MOEs, 
and  where  a  function  is  a  process  relating  in  an  outcome  [Ref  10].  Thus  "an  MOE 
defined  by  an  objective  function  at  an  upper  level  is  a  dependent  variable,  and  is  a 
mathematical  function  of  the  MOEs  defined  by  objective  functions  at  a  lower  level."  [Ref. 
10]  Ultimately,  an  "audit  trail"  equation  is  generated,  linking  the  conditional  upper  level 
MOE  to  measurable  MOPs.  Elele  uses  Bayes'  Rule  to  develop  a  similar  probability  based 
MOE  definition  [Ref  6]. 

At  an  even  higher  level  of  abstraction,  Levis  and  Cothier  [Ref  11],  Levis  [Ref 
12],  and  Levis  and  Martin  [Ref  13]  have  developed  an  MOE  methodology  based  on  set 
theory.  Similarly,  Dockery  [Ref  14]  explores  fuzzy  set  theory  in  defining  MOEs. 
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d.  Measures  of  Force  Effectiveness 

An  MOFE  is  a  measure  of  how  the  force  performs  its  missions,  or  its 
"contribution  to  battle  outcome."  [Ref  7]  Again,  MOFEs  are  inherently  scenario 
dependent,  since  they  are  tied  to  operational  context  and  assumed  enemy  actions. 

B.  C^I  SYSTEM  EVALUATION  TECHNIQUES 

1.  Cost  and  Operational  Effectiveness  Analysis  (COEA) 

The  MOEs  and  doctrinal  framework  presented  later  in  chapter  HI  will  be  developed 
primarily  for  use  with  the  cost  and  operational  effectiveness  analysis.  A  COEA  is  a 
systematic  study  "designed  to  assist  a  decision  maker  in  identifying  a  preferred  choice 
among  possible  alternatives."  [Ref.  15]  COEAs  are  currently  the  tool  of  choice  for  the 
evaluation  of  alternative  systems  during  the  concept  development  phase  of  acquisition. 

The  first  step  in  a  COEA  involves  defining  the  overall  objectives  of  the  study, 
identifying  a  "base  case"  system  (usually  consisting  of  currently  fielded  assets),  and  a  set  of 
alternative  systems  to  be  evaluated.  The  evaluation  itself  usually  focuses  on  three  areas; 
effectiveness,  cost,  and  risk,  of  which  effectiveness  requires  the  development  of  MOEs 
that  can  be  applied  equally  to  the  base  case  and  each  alternative.  Figure  3  shows  a  normal 
COEA  methodology. 
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2.  Modular  Command  and  Control  Evaluation  System  (MCES) 

The  modular  command  and  control  evaluation  system  was  developed  by  a  team  of 
experts  from  industry,  government  and  academia,  in  a  series  of  forums  initiated  by  the 
MITRE  Corporation  and  the  Military  Operations  Research  Society,  and  hosted  by  the 
Naval  Postgraduate  School  during  the  1980s.  The  purpose  of  the  forums  was  to  develop 
a  general  approach  to  analyzing  systems,  and  the  result  of  their  efforts  is  a  seven 
module  system  that  has  been  successfully  applied  to  many  facets  of  C^,  including  planning. 
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acquisition,  testing  and  operation  [Ref.  16],  The  strength  of  the  MCES  approach  lies  in 
the  guidance  it  provides  to  analysts  who  might  otherwise  focus  prematurely  on  the 
quantitative  model,  rather  than  the  problem  definition  and  the  specific  measures  needed  to 
discriminate  between  alternatives  [Ref  16],  The  MCES  presents  a  method  to  attack 
difficult  concepts  in  a  standardized  manner,  using  some  semblance  of  standardized 
terminology  and  paradigms.  Ultimately  the  MCES  can  be  thought  of  as  two  processes,  a 
managerial  system  which  serves  as  a  guide  to  specifying  the  problem  to  be  analyzed,  and 
an  analytic  system  which  serves  as  a  guide  to  the  analysis  process  itself  [Ref.  17].  The 
seven  modules  are  described  individually  below. 

The  first  module,  problem  formulation,  identifies  the  objective  of  an  application, 
leading  to  a  precise  statement  of  the  problem  being  addressed.  The  system's  operational 
and  deployment  concepts,  environmental  factors,  scenarios,  assumptions  and  threats  are 
made  explicit.  [Ref  17] 

Module  two,  system  bounding,  takes  the  output  of  the  first  module  (the  problem 
statement),  and  uses  it  to  bound  the  system.  The  system  bounds  are  defined  in  terms  of 
the  three  categories  of  a  C^I  system  (defined  earlier  in  the  chapter  as  physical  entities, 
structures,  and  processes),  and  the  onion  skin  diagram.  The  implementation  of  this 
module  results  in  the  identification  and  categorization  of  the  system  elements.  [Ref  17] 

Module  three,  process  definition,  involves  building  a  dynamic  framework  that 
identifies  the  relevant  C^  processes,  each  broken  doAvn  into  a  set  of  functions.  The 
framework  focuses  attention  on  the  environmental  forces  which  initiate  the  process,  the 
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internal  process  functions  which  detail  what  the  system  is  doing,  and  the  required  input 
to  and  output  from  the  internal  process.  The  implementation  of  this  module  results  in 
the  identification  of  the  functions  of  the  process,  which  are  mapped  into  a  process 
loop.  The  generic  process  loop  consists  of  an  environmental  stimulus,  which  initiates  a 
process  of  sense,  assess  (with  regard  to  desired  state),  generate,  select,  plan,  and  direct, 
the  result  of  which  is  a  system  response  to  one's  own  forces,  inside  the  context  of  the 
environment.  [Ref  17] 

The  fourth  module  of  the  process,  integration  of  system  elements  and  functions, 
integrates  the  system  elements  identified  in  module  two,  and  the  process  functions  fi'om 
module  three,  into  a  representation  of  the  Cl  system.  The  product  of  this  module  is  at 
least  a  complete  descriptive  conceptual  model,  and  perhaps  a  complete  mathematical 
model.  [Ref  17] 

Using  the  four  prior  modules  as  a  base,  the  fifth  module,  specification  of  measures, 
specifically  identifies  the  MOPs,  MOEs,  and  MOFEs  necessary  to  address  the  problem  of 
interest.  The  measures  are  specific  with  regard  to  the  corresponding  levels  of  the  C^I 
system  and  C"*!  functions.  Resulting  measures  must  adhere  to  a  set  of  desired 
characteristics,  which  often  includes  whether  the  measure  is  mission-oriented, 
discriminatory,  measurable,  objective,  sensitive,  and  inclusive  [Ref  17],  Note  that  the 
MCES  can  be  utilized  just  for  the  implementation  of  the  first  five  modules,  which  will 
result  in  a  concise  problem  statement,  a  complete  description  of  the  C^I  system,  including 
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its  functions,  and  a  list  of  acceptable  measures.  The  set  of  measures  can  then  be  applied  in 
the  form  of  another  evaluation  methodology,  such  as  a  COEA. 

Module  six,  data  generation,  involves  the  generation  of  data  for  the  measures 
identified  in  the  previous  module.  Any  appropriate  type  of  data  generator  may  be  used, 
including  simulation,  exercises,  experiments,  or  expert  opinion  [Ref  17].  The  module 
output  is  simply  the  values  associated  with  the  measures  of  interest. 

The  final  module,  aggregation  of  measures,  aggregates  and  interprets  the  observed 
values  of  the  measures.  The  analysis  results  are  tailored  to  address  the  problem  statement 
of  module  one.  The  seven  MCES  modules  are  performed  iteratively,  with  the  decision 
maker  located  as  shown  in  Figure  4.  [Ref  17] 
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Figure  4.  MCES  Structure.  From  Ref.  [17]. 
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3.  Mission  Oriented  Approach 

The  mission  oriented  approach  is  a  four  phase  framework  for  formulating 
requirements  while  maintaining  an  optimal  balance  between  operational  requirements, 
technical  capabilities,  and  scarce  resources  (see  Figure  5)  [Ref  18],  The  MOA 
"systematically  relates  mission  objectives  and  operational  plans  to  system  objectives 
and  associated  programs."  [Ref  18]  The  approach  is  somewhat  similar  to  the  MCES,  in 
that  the  first  two  phases  of  MOA  relate  to  the  functional  definition  module  of  MCES,  and 
the  last  two  phases  of  MOA  deal  with  the  capabilities  in  MCES  module  four.  Like  the 
MCES,  the  MOA  is  beneficial  as  a  potentially  common  framework  for  discussions  among 
communities,  although  it  lacks  the  commonalty  of  terms  built  into  MCES.  Finally,  while 
the  MOA  is  well  suited  for  requirements  definition,  it  is  not  particularly  well  suited  for 
evaluation  of  alternative  C'*!  systems  [Ref  19] 

The  first  phase  begins  by  addressing  the  question  "What  are  we  trying  to  achieve 
operationally?"  This  question  is  directed  well  above  the  operator  and  analyst  level,  to  the 
relevant  political  and  military  decision  makers.  The  response  to  this  question  is  generally 
framed  in  terms  of  a  set  of  strategic  capability  objectives  for  employing  forces.  These 
force  capability  objectives  become  the  standard  against  which  the  capabilities  of  existing, 
and  proposed,  "packages"  of  C^  systems  can  be  measured.  [Ref  18] 
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Figure  5.  Generic  MO  A.  From  Ref.  [18], 
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Phase  two  addresses  the  related  question,  "How  are  we  trying  to  perform  the 
mission?"  This  question  should  be  answered  by  those  operational  personnel  who 
formulate  concepts  of  operation  at  multiple  levels;  t)qDical  levels  of  aggregation  include 
strategic  capability,  mission  (operational)  capability,  and  functional  capability.  For  each 
level,  the  capability  objectives  are  "self-consistently  derived,  beginning  with  specified 
strategic  capability  levels,  based  on  perceived  adversary  operations,  fiiendly  concepts  of 
operation  and  environmental  factors."  [Ref  18] 

The  third  phase  of  the  MOA,  addresses  "what  technically"  is  required  to  support  the 
operation  defined  in  the  first  two  phases.  Technical  personnel  are  employed  to  "translate 
the  operational/mission/fimction  capability  objective  levels  into  the  technical  attributes  of 
the  packages  of  communications"  needed  to  implement  the  respective  capability  levels 
[Ref  18].  Technical  characteristics  are  derived  using  the  existing,  and  proposed, 
characteristics  of  fiiendly  forces,  enemy  forces  and  the  environment. 

Having  defined  "what  technically"  is  required  in  the  third  phase,  the  fourth  phase  of 
the  process  describes  "how  technically"  will  the  desired  characteristics  be  incorporated. 
Based  on  technical  deviations  obtained  by  comparing  existing  packages  of  systems  to 
the  technical  objectives  of  phase  three,  alternative  packages  of  systems  can  be 
formulated  "consistent  with  assumed  restrictions  on  available  resources."  [Ref  18]  Thus 
potential  "investment  strategies"  are  outlined,  which  can  then  be  assessed  to  determine 
how  well  they  meet  the  strategic  objectives  set  forth  by  policy  makers  in  phase  one  [Ref 
18],  Note  that  by  going  back  to  phase  one,  the  MOA  becomes  an  iterative  process. 
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indeed  "the  key  to  successful  application  of  MOA  is  iteration."  [Ref  18]  The  iteration  is 
needed  to  address  issues  across  communities  (e.g.,  operational  and  technical),  and  within 
communities  (e.g.,  system  operators  and  maintainers). 

4,  Headquarters  Effectiveness  Analysis  Tool 

The  headquarters  effectiveness  evaluation  tool,  or  HEAT,  is  based  on  adaptive 
control  system  theory  [Ref  20].  Two  vari--  'ts  of  HEAT  are  currently  in  use,  they  are  the 
Army  command  and  control  evaluation  system  (ACCES),  and  a  Navy  version  for  battle 
force  in-port  training  (BFIT)  [Ref  21].  The  primary  focus  of  HEAT  is  the  overall 
performance  of  a  headquarters,  and  in  general  HEAT  addresses  the  question  "What  should 
be  done  to  improve  the  effectiveness  of  a  military  headquarters?"  [Ref  21]  ACCES 
focuses  this  question  primarily  on  the  division  headquarters  [Ref  20].  Candidates  for 
improvement  include  the  headquarters  staff  (staff  size,  organization,  training,  procedures, 
etc.),  and  C'*!  systems.  In  execution,  HEAT  uses  specific  definitions  of  Cl  theory,  tools 
and  applications. 

Theory,  in  the  form  of  a  decision-making  model,  aids  in  the  design  of  measurement 
tools,  and  suggests  hypotheses  for  experiments  and  exercises.  The  decision  making  model 
is  "the  most  fundamental  concept  in  HEAT  theory."  [Ref  21]  The  model  itself  is  an 
expanded  Boyd  cycle,  and  includes  the  six  "process  steps"  of  monitoring  (obtaining  data 
from  the  environment),  estimation  (placing  values  on  monitoring),  situation  assessment 
(interpreting  estimates),  option  generation  (generates  feasible  courses  of  action),  option 
selection,  and  plan-generation/direction  [Ref  21].  The  model  "asserts  that  planning, 
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decisionmaking,  and  execution  activities  conducted  by  a  headquarters  are  analogous  to  a 
feedback  control  system  that  attempts  to  keep  its  external  environment  within  acceptable 
bounds,  despite  the  actions  of  the  enemy."  [Ref  21],  The  six  process  steps  transcend 
organizational  boundaries,  since  each  step  does  exist  (in  some  form)  in  virtually  every 
headquarters,  making  the  model  a  foundation  for  posing  and  answering  questions  about 
headquarters  effectiveness. 

The  HEAT  measurement  tool  consists  of  a  set  of  measures  that,  as  a  whole,  capture 
important  determinants  of  effectiveness.  ACCES  in  particular,  has  forty-three  MOEs 
designed  to  determine  command  post  effectiveness  [Ref  20],  The  HEAT  measures  fall 
into  three  broad  categories:  process,  performance,  and  effectiveness.  Process  and 
performance  measures  can  be  further  defined  under  four  types:  characteristic  measures 
(process  measures  that  indicate  whether  a  headquarters  strives  to  be  optimal), 
coordination  measures  (process  measures  that  capture  the  degree  of  information 
coherence),  queueing  measures  (performance  measures  that  indicate  the  origins  of  time 
delays),  and  quality  measures  (performance  measures  that  indicate  the  degree  to  which 
performance  approaches  a  norm  -  usually  the  accuracy  of  the  headquarters  perception  of 
ground  truth)  [Ref  21].  HEAT  MOEs  that  may  be  pertinent  to  Marine  C^I  systems  are 
included  in  Chapter  III  of  this  paper. 

A  HEAT  application  results  in  the  assignment  of  values  to  the  HEAT  measures,  and 
may  consist  of  experiments  and/or  exercises.  Note  that  ecperiments  and  exercises  are 
complementary,  as  exercises  can  help  validate  experiment  findings,  or  suggest  independent 
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variables  for  future  experiments.  A  key  aspect  of  a  HEAT  application  is  the  requirement 
for  observation.  Towards  this  end,  HEAT  observers  must  undergo  specialized  training, 
which  includes  fundamental  HEAT  concepts  and  the  rudimentaries  of  data  collection  [Ref 
21]. 

Although  the  HEAT  methodology  is  well  defined,  and  has  been  successfully  applied 
by  analysts,  the  reliance  on  a  defined  set  of  MOEs  limits  this  evaluation  technique  when 
compared  to  the  MCES  or  the  MOA.  Additionally,  HEAT  uses  a  standardized  model,  in 
contrast  to  the  formal  problem  and  environmental  definitions  found  in  the  initial  modules 
of  the  MCES.  The  Army  has  found  that  "the  most  debilitating  problem  with  the  ACCES 
methodology  is  the  reliance  upon  percentage  figures  ...  raw  scores  would  provide  a  great 
deal  more  precise  and  reliable  information."  [Ref  20] 

5.  Evolutionary  Upgrade  Paths 

Most  existing  C'*!  systems,  especially  recent  procurements,  will  likely  be 
incorporated  with  some  type  of  product  improvement  or  upgrade  during  their  useful  life 
cycle.  The  concept  of  evolutionary  upgrade  paths  is  that  an  effective  evaluation  of  a  new 
O'*!  system  must  include  an  evaluation  of  its  planned  upgrade  path  toward  some  goal  or 
target  level  of  functionality.  The  evolutionary  acquisition  concept  is  a  "build  a  little,  test  a 
little,  field  a  little"  approach,  using  off-the-shelf  equipment  and  software  where  applicable. 
More  formally,  it  is  "an  acquisition  strategy  which  may  be  used  to  procure  a  system 
(which  is)  expected  to  evolve  during  development,  within  an  approved  architectural 
framework,  to  achieve  an  overall  systems  capability."  [Ref  19] 
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Proponents  of  evolutionary  acquisition  point  to  several  compelling  reasons  for 
utilizing  the  approach.  Past  attempts  at  using  the  "big  system"  approach,  that  is,  fielding  a 
complete,  integrated  system  all  at  once,  have  been  somewhat  unsuccessful.  Examples  of 
perceived  failures  include  the  many  iterations  of  TCO  (starting  in  1968),  and  MIFASS 
[Ref  19].  Additionally,  it  is  difficult  to  generate  a  complete  list  of  defined  requirements 
for  a  needed  system;  new  technology,  new  procedures,  and  the  length  of  the  acquisition 
process  all  add  to  the  uncertainty  of  predicting  the  final  system  requirements.  Finally, 
proponents  argue  that  user  response  is  more  quickly  incorporated  and  capabilities  are 
fielded  faster  using  evolutionary  acquisition.  By  starting  with  a  fielded  system  with  which 
the  user  is  already  familiar,  the  user  can  better  assimilate  and  evaluate  changes.  Also,  by 
quickly  fielding  a  core  capability  while  concurrently  developing  component  systems,  new 
technology  (including  commercial-off-the-shelf  items)  can  reach  the  user  at  a  rate  much 
faster  than  is  currently  possible. 

The  evolutionary  upgrade  framework  consists  of  four  major  steps.  Step  one  is  to 
define  the  target  system's  functions  and  capabilities.  The  system  capability  requirements 
are  detailed  in  the  originating  mission  need  statement,  and  the  operational  requirements 
document.  Similarly,  O'*!  functions  are  well  documented  in  doctrine,  as  well  as  the 
previously  mentioned  Marine  Corps  Mission  Area  11  analysis,  although  functional 
decompositions  may  be  required  to  meet  the  desired  resolution.  Once  defined,  functions 
must  be  matched  to  the  technological  capabilities  required  to  support  them.  With  the 
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relationship  between  target  system  capabilities  and  functions  established,  weights  are 
assigned  to  the  function  according  to  perceived  value. 

The  second  step  is  to  define  all  viable  upgrade  paths.  For  each  potential  base 
system,  the  goal  is  to  establish  a  list  of  candidate  paths  which  result  in  a  target  system  that 
possesses  the  capabilities  established  in  the  original  definition  of  the  target  system.  Base 
systems  should  preferably  meet  some  of  the  requirements  of  the  target  system,  and  may 
have  multiple  candidate  paths.  The  underlying  concept  is  that  each  base  system  could 
someday  meet  the  total  requirements  of  the  target  system,  by  incrementally  adding  Titure 
capabilities.  Finally,  each  candidate  path  should  satisfy  the  current  cost,  resource,  and  risk 
constraints. 

The  third  step  of  the  framework  involves  determining  the  overall  value  and  cost  of 
each  candidate  path.  The  values  for  each  function  were  determined  in  step  one,  and  by 
summing  the  values  of  the  functions  met  while  incorporating  some  "discount"  function  to 
account  for  the  timeliness  with  which  the  base  system  reaches  target  system  maturity.  The 
time  series  approach  to  discounting  is  necessary  since,  by  definition,  each  candidate  path 
ultimately  incorporates  each  of  the  functions  required  in  the  target  system,  which  would 
cause  the  sum  of  the  function  values  for  each  path  to  be  equal.  Additionally,  the  concept 
of  "functionally  derived  values,"  or  attaching  greater  values  to  the  most  important 
technological  capabilities,  can  provide  valuable  insights  into  system  priorities.  [Ref  19] 

The  cost  of  each  candidate  path  can  be  computed  by  adding  the  cost  of  the 
alternative  base  system  if  bought  today,  and  the  cost  of  adding  a  new  capability  in  the 
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future.  Note  that  the  costs  to  achieve  the  target  system  may  differ  between  base  systems, 
due  to  the  potential  differences  in  hardware,  software,  and  configurations.  Costs  should 
be  put  in  constant  dollars,  and  should  include  costs  associated  with  research,  development, 
testing,  and  evaluation  (RDT&E),  procurement,  and  operations  and  support  costs.  Like 
values,  costs  can  be  "discounted"  to  account  for  technological  priorities.  [Ref  19] 

The  final  step  can  be  thought  of  as  a  linear  program,  where  one  is  maximizing  value, 
subject  to  cost,  resource,  and  risk  constraints.  The  final  selection  method  would  involve 
the  elimination  of  those  candidate  paths  that  violated  the  constraints,  and  selecting  from 
the  remaining  paths  the  one  with  the  highest  value. 

6.  Functional  Decompositions 

To  an  extent,  some  form  of  functional  decomposition  is  present  in  most  of  the 
preceding  techniques,  such  as  the  detailed  user  requirements  in  the  COEA,  module  three 
in  the  MCES,  or  "how  operationally"  in  the  MOA.  Several  less  used  O'*!  evaluation 
techniques  employ  variations  of  functional  decomposition  as  the  primary  framework  for 
their  methodology. 

Kemple,  Stephens,  and  Crolotte  [Ref  22]  use  expert  opinion  in  a  three  part 
methodology  that  defines  MOEs  for  C**!  systems.  First,  a  three  phase  "value  analysis'*  is 
conducted  where  experts  define  "attributes"  of  system  effectiveness,  weight  them  by 
relative  importance,  and  then  reduce  them  to  a  reasonably  sized  set  of  the  most  significant 
attributes.  Next  a  functional  analysis  is  conducted,  using  a  system  fimctional 
decomposition  which  is  again  weighted  by  experts,  in  order  to  facilitate  comparisons. 
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Finally,  the  value  and  functional  analyses  are  combined  to  generate  what  should  be 
measured,  appropriate  scenarios,  and  MOEs  defined  through  linear  aggregation.  [Ref  22] 
Girard  [Ref  10],  has  developed  what  he  calls  "a  function  based  definition  of 
MOEs."  Using  requirements  documents,  such  as  the  Navy's  Top  Level  Warfare 
Requirement  (TLWR),  a  functional  decomposition  is  conducted  in  each  of  three  domains: 
mission,  organization,  and  resource.  Mission  success  criteria  (MSCs)  and  required 
capabilities  (RCs),  both  established  by  the  TLWR,  are  used  to  generate  the  initial  tiers  of 
functions.  Most  mission  success  criteria  and  required  capabilities  can  be  stated  in  terms  of 
the  probability  of  successfully  accomplishing  a  function,  therefore  a  function  defines  its 
own  metric.  Conversely,  a  probability  metric  defines  the  outcome  of  its  related  function. 
For  example,  the  probability  of  detection  identifies  the  detection  event  as  the  outcome  of 
the  search  function;  functions,  and  their  related  metrics,  become  inseparable.  [Ref.  10] 

RCs  can  be  thought  of  as  a  set  of  subfimctions  for  the  higher  level  functions  defined 
by  the  MSCs.  In  this  way,  a  procedure  (a  set  of  functions  carried  out  in  some  prescribed 
manner)  represents  the  implementation  of  the  higher  order  function  in  terms  of  its 
subfunctions.  A  model  of  the  procedure  will  be  nothing  more  than  a  conditional 
probability  describing  the  relationship  of  the  lower  level  metrics  to  the  higher  level  metrics. 
In  this  manner,  Grirard  arrives  at  his  definition  of  an  MOE  as  the  (conditional)  probability 
of  the  successful  accomplishment  of  a  function  [Ref  10].  The  MOEs  resulting  from  the 
decomposition  can  then  be  applied  using  a  variety  of  analytical  tools. 
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The  MOE  Handbook  issued  by  the  Army's  Training  and  Doctrine  Command 
(TRADOC)  also  uses  functional  decomposition  methodology  to  describe  a  method  for 
developing  MOEs.  As  a  first  step,  the  methodology  lists  six  components  which  must  be 
defined:  objectives,  functions,  processes,  tasks,  structures,  and  physical  entities.  These 
components  represent  "links"  between  the  C^'I  system,  the  decisionmaker's  questions,  and 
the  hierarchy  of  O'*!  components.  A  rough  outline  of  the  links  can  help  portray 
interrelationships  and  implications  inherent  in  the  decisionmaker's  questions.  The 
questions  themselves  should  be  simplified,  with  the  goal  being  "to  break  those  high  level 
questions  down  through  a  systematic  functional  decomposition  that  results  in  the 
development  of  issues  or  questions  that  are  specifically  worded  to  obtain  data."  [Ref  23]. 
Through  the  functional  decomposition  of  the  decisionmaker's  questions,  issues,  sub-issues, 
essential  elements  of  analysis,  MOFEs.  MOEs  MOPs,  and  data  requirements  are 
developed  which  guide  and  direct  the  required  analytical  work.  The  handbook 
recommends  a  logic  tree  (dendritic  network)  to  portray  the  interconnectivity  of  a  logical 
network  used  in  decomposing  the  decisionmaker's  questions  into  issues.  [Ref  23] 

C.  SUMMARY 

Although  the  COEA  is  the  standard  tool  employed  by  the  Marine  Corps  to  evaluate 
alternative  C^I  systems  during  the  concept  definition  phase  of  acquisition,  this  chapter  has 
suggested  that  there  are  several  equally  robust  evaluation  tools.  Significantly,  alternate 
evaluation  techniques  can  be  complementary  to  the  traditional  COEA  firamework.  An 
application  of  one  of  the  functional  decomposition  techniques  can  strengthen  the 


31 


originating  requirements  documents,  the  MNS  and  the  ORD,  The  MCES,  with  its 
commonalty  of  terms  and  robust  problem  definition,  can  aid  in  the  identification  of 
measures  that  will  provide  a  valid  comparison  between  the  alternative  systems  in  the 
COEA.  Finally,  the  evolutionary  upgrade  path  framework  can  add  resolution  to 
alternative  systems  that  in  the  current  COEA  framework  might  otherwise  "be  too  close  to 
call." 
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ni.  roENTIFICATION  AND  ANALYSIS  OF  MEASURES 


A.  BACKGROUND 

Due  to  the  variety  of  functions  different  systems  may  perform,  the  MOEs  to  be 
used  in  an  analysis  must  often  be  devised  specifically  for  the  task  at  hand.  Indeed,  the 
MCES  methodology  makes  clear  that  prior  to  identifying  MOEs,  the  problem  must  be 
clearly  stated  and  the  environment  must  be  fully  defined.  As  has  already  been  mentioned, 
an  MOE  at  one  level  might  be  an  MOFE  or  an  MOP  at  another  level. 

Many  good  MOEs,  however,  are  already  in  existence,  and  there  is  no  requirement  to 
be  original  in  identifying  MOEs  for  a  particular  study.  Pertinent  MOEs  may  be  gleaned 
from  past  analyses,  studies,  and  academic  submissions  to  those  periodicals  that  deal  with 
command  and  control  (e.g..  Signal  Magazine  and  IEEE).  When  dealing  with  the  large 
body  of  existing  MOEs,  the  primary  problem  is  determining  which  are  applicable  and 
valid.  Further,  the  sheer  volume  of  existing  MOEs  requires  some  sort  of  framework  to 
categorize  them;  a  workable  framework  allows  manageable  comparisons  to  be  made 
between  potential  alternatives.  Several  potential  fi^meworks  present  themselves  for 
consideration,  including  categorizing  by  system  composition  (hardware,  software, 
communications  equipment),  system  functions,  or  system  cost.  A  system  based 
categorization,  however,  fails  to  narrow  the  set  of  existing  MOEs  with  regard  to  Marine 
Corps  C^I  directives.  To  that  end,  existing  USMC  doctrine  pertinent  to  the  C*!  arena  was 
reviewed  in  order  to  gain  insight  into  a  framework  with  more  specific  application  to  die 
Marine  Corps.  This  framework  will  be  summarized  in  the  sections  that  follow. 
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B.  DOCTRINAL  FRAMEWORK 


Marine  Corps  doctrine  concerning  command  and  control,  while  voluminous,  is  also 
very  specific  in  regard  to  required  capabilities  and  characteristics.  FMFM  3,  Command 
and  Control,  provides  a  structural  framework  for  the  classification  of  C^  system  ideas, 
requirements,  capabilities  and  characteristics  listed  in  other  doctrine,  including  FMFM 
3-30  Communications,  Mission  Area  Analysis  of  Mission  Area  11  -  Command  and 
Control,  FMFRP  14-30  A  Concept  of  Command  an*.-  Control,  and  The  Command  and 
Control  Concept  Final  Report.  FMFM  3  is  particularly  pertinent  because  it  marks  "the 
first  (Marine  Corps)  effort  to  apply  command  and  control  theory  specifically  to  maneuver 
warfare  in  doctrinal  terms."  [Ref  24]  Specifically,  the  framework  suggested  by  FMFM  3 
consists  of  five  broad  categories,  which  seem  to  be  inclusive  of  the  crucial  C'*!  system 
requirements  discussed  throughout  Marine  Corps  doctrine.  The  doctrinal  categories, 
taken  directly  from  FMFM  3,  are  as  follows. 

1)  Facilitate  Commander's  Influence  of  Events 

2)  Adapt  to  the  Situation 

3)  Support  Information  Requirements 

4)  Exploit  MAGTF  Capabilities 

5)  Support  CP  Effectiveness 

Broader  descriptions  of  these  categories  are  given  in  the  sections  that  follow. 

In  order  to  assess  the  extent  to  which  these  doctrinal  categories  encompass  all  Marine 
Corps  C^I  doctrinal  requirements,  and  to  describe  the  doctrinal  categories  in  greater  detail. 
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additional  Marine  Corps  O'*!  publications  were  examined  for  further  elements  of  doctrine. 
Any  statement  that  defines  additional  C'*!  performance  or  capability  requirements  was 
listed  as  an  element  of  doctrine.  Table  1  shows  examples  of  these  additional  elements  of 
doctrine,  which  total  39  statements  from  six  publications. 


A  system  "places  decision  making  at  the  lowest  possible  level." 

A  system  "ensures  subordinate  commanders  receive  ideas  and  concepts,  not  just  data  packages." 

A  system  should  "translate  the  commander's  decisions  into  plans  and  orders." 

A  system  should  "supervise  the  execution  of  those  plans  and  orders." 

"The  MAGTF  commander  must  be  able  to  make  and  implement  decisions  faster  than  the  enemy,  so  as  to 
gain  the  initiative  and  create  tactical  advantages  for  the  MAGTF  to  exploit." 

A  system  should  possess  the  "capability  for  MEF  and  MEB  command  elements  to  conduct  rapid 
mission  planning  at  a  level  comparable  to  that  of  a  MEU." 

Table  1.  Examples  of  Additional  Elements  of  Doctrine. 

The  additional  elements  of  doctrine,  hereafter  referred  to  as  "supporting  elements," 
were  placed  into  the  context  of  the  five  doctrinal  categories  in  order  to  determine  the 
comprehensiveness  of  the  fi'amework.  The  actual  placement  of  supporting  elements  into 
the  doctrinal  categories  was  based  on  the  author's  determination  of  whether  the  supporting 
elements  reinforced  or  clarified  the  description  of  a  particular  category.  For  example,  the 
definition  of  doctrinal  category  one,  "Facilitate  Commander's  Influence  of  Events,"  is  as 
follows. 


No  coherent  battle  plan  is  possible  without  the  commander's  vision  of 
how  he  intends  to  win.  The  system,  imbued  by  his  intent,  provides  the 
means  to  apply  his  decision  effectively,  gather  and  analyze  information, 
implement  his  plan,  and  supervise  his  forces  [Ref  24]. 
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Thus  the  supporting  element  "translate  the  commander's  decision  into  plans  and  orders" 
reinforces  that  portion  of  the  doctrinal  category  definition  which  discusses  the  need  for  a 
"means  to  apply  his  decision  effectively."  Similarly,  the  supporting  element  which  states 
that  a  Cl  system  should  provide  "the  capability  to  view  friendly  and  enemy  force  data  as  it 
is  received"  clarifies  (and  provides  resolution  to)  the  category  definition  assertion  that  a 
C^l  system  should  help  a  commander  "supervise  his  forces." 

In  this  manner,  a  subset  of  the  29  supporting  elements  were  determined  to  fit  within 
the  first  doctrinal  category;  Table  2  reflects  this  subset.  The  same  logic  was  applied  to  fit 
supporting  elements  within  the  remaining  four  doctrinal  categories,  with  some  supporting 
elements  appearing  in  more  than  one  doctrinal  category.  Ultimately,  all  39  supporting 
element  statements  were  successfully  placed  into  the  doctrinal  framework,  suggesting  that 
the  framework  is  indeed  inclusive  of  all  Marine  Corps  C*I  doctrine.  A  summary  of  the 
distribution  of  supporting  elements  under  each  doctrinal  category  is  included  in  the 
sections  that  follow. 

Once  the  inclusiveness  of  the  doctrinal  categories  was  established,  MOEs  selected 
from  the  literature  were  mapped  into  the  doctrinal  categories  of  the  framework,  and 
further  related  to  the  supporting  elements  already  listed  under  each  doctrinal  category; 
the  supporting  elements  were  used  to  categorize  the  MOEs  because  they  provided 
increased  resolution  to  the  doctrinal  category  definitions.  The  increased  resolution,  in 
turn,  was  deemed  necessary  due  to  the  specificity  of  many  of  the  MOEs.  Like  supporting 
elements,  MOEs  were  put  into  the  framework  based  on  the  author's  interpretation  of  their 
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relationship  to  the  categories.  This  interpretation  was  based  on  the  stated  definition  of 
each  MOE,  as  well  as  their  respective  statements  of  purpose. 


A  system  "places  decision  making  at  the  lowest  possible  level." 

A  system  "ensures  subordinate  commanders  receive  ideas  and  concepts,  not  just  data  packages. 

AC  system  should  “translate  the  commander's  decisions  into  plans  and  orders." 

AC*  system  "supervise  the  execution  of  those  plans  and  orders." 

"The  MAGTF  commander  must  be  able  to  make  and  implement  decisions  faster  than  the  enemy,  so  as 
to  gain  the  initiative  and  create  tactical  advantages  for  the  MAGTF." 

A  c*  system  should  possess  the  "capability  for  MEF  and  MEB  command  elements  to  conduct  rapid 
mission  planning  at  a  level  comparable  to  that  of  a  MEU." 

AC*  system  should  possess  the  "capability  to  view  friendly  and  enemy  force  data  as  it  is  received." 

A  c*  system  should  possess  the  "capability  to  complement  the  commander's  situational  awareness  and 
instincts." 

"Support  planning  for  future  operations  by  offering  the  commander  the  ability  to  gain  or  create  an  image 
of  the  battlefield  at  ranges  in  time  and  distance  beyond  the  scope  of  current  operations." 

Table  2.  Subset  of  Supporting  Elements  for  First  Doctrinal  Category. 

Sections  1  through  5  below  present  the  details  of  the  framework,  with  each  section 
titled  by  the  doctrinal  category  to  be  presented.  Following  the  title  is  the  definition  of  the 
doctrinal  category.  This  definition  is  itself  followed  by  the  header  "supporting  elements," 
under  which  are  listed  the  supporting  elements  included  in  the  doctrinal  category.  Each 
supporting  element  is  preceded  by  a  lower  case  letter  (in  alphabetical  order,  starting  with 
(a)),  which  will  be  used  to  reference  the  respective  supporting  element  in  Tables  3  through 
7. 


37 


The  table  which  follows  the  list  of  supporting  elements  in  each  section  portrays  the 
MOEs  included  in  the  doctrinal  category.  Next  to  each  MOE,  an  "X"  is  recorded  below 
the  letter  of  each  supporting  element  that  it  measures.  For  example,  in  Table  3,  the  MOE 
"Number  of  Orders  Issued"  can  be  used  to  measure  supporting  elements  (a)  ("places 
decision  making  at  the  lowest  possible  level"),  and  (b)  ("ensures  subordinate  commanders 
receive  ideas  and  concepts,  not  just  data  packages").  An  empirical  indication  of  MOE 
performance  within  a  doctrinal  category  can  be  determined  by  counting  the  number  of 
supporting  elements  it  measures.  A  full  description  of  each  MOE,  including 
computational  methods  and  references,  is  included  in  Appendix  A. 

As  part  of  this  research,  the  MOEs  within  each  doctrinal  category  are  ranked 
according  to  the  extent  to  which  each  provides  a  realistic  assessment  of  how  well  a  Cl 
system  meets  the  stated  O'*!  system  requirements  within  the  doctrinal  category.  The 
rankings  for  the  MOEs  within  each  doctrinal  category  are  included  in  the  last  column  of 
each  table.  Rankings  were  accomplished  using  expert  opinion  garnered  by  a 
questionnaire.  Questionnaires  were  completed  by  twelve  respondents,  including  eleven 
Marine  Officers  and  one  NFS  Operations  Research  Department  professor.  The  ranked 
lists  were  analyzed  using  a  variant  of  the  Method  of  Successive  Interval  Scaling,  where 
experts  are  asked  to  rank  instances  in  terms  of  how  much  of  a  property  they  possess  [Ref 
25].  The  questionnaire  and  resulting  calculations  are  included  in  Appendix  B. 
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1.  Doctrinal  Category:  Facilitate  Commander's  Influence  of  Events 

"No  coherent  battle  plan  is  possible  without  the  commander's  vision  of  how  he 
intends  to  win.  The  system,  imbued  by  his  intent,  provides  the  means  to  apply  his 
decision  effectively,  gather  and  analyze  information,  implement  his  plan,  and  supervise  his 
forces."  [Ref  24] 

0.  Supporting  Elements 

(a)  A  C^  system  "places  decision  making  at  the  lowest  possible  level."  [Ref  8] 

(b)  A  system  "ensures  subordinate  commanders  receive  ideas  and  concepts, 
not  just  data  packages."  [Ref  8] 

(c)  A  system  should  "translate  the  commander's  decisions  into  plans  and 
orders."  [Ref  26] 

(d)  A  C^  system  should  "supervise  the  execution  of  those  plans  and  orders." 

[Ref  26] 

(e)  "The  MAGTF  commander  must  be  able  to  make  and  implement  decisions 
faster  than  the  enemy,  so  as  to  gain  the  initiative  and  create  tactical  advantages  for  the 
MAGTF  to  exploit."  [Ref  1] 

(f)  Should  possess  the  "capability  for  MEF  and  MEB  command  elements  to 
conduct  rapid  mission  planning  at  a  level  compilable  to  that  of  a  MEU."  [Ref  1] 

(g)  Should  possess  the  "capability  to  view  friendly  and  enemy  force  data  as  it  is 
received."  [Ref  1] 
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(h)  Should  possess  the  "capability  to  complement  the  commander's  situational 


awareness  and  instincts."  [Ref,  1] 

(i)  "Support  planning  for  future  operations  by  offering  the  commander  the 
ability  to  gain  or  create  an  image  of  the  battlefield  at  ranges  in  time  and  distance  beyond 
the  scope  of  current  operations."  [Ref  1] 


Measure  of  Effectiveness 

Measures  These  Supporting 

Elements 

Performance 

(a) 

(b) 

(C) 

(d) 

(e) 

(0 

(g) 

(h) 

(i) 

System  contribution  to  the  commanders 

perception  of  the  enemy. 

X 

X 

X 

2 

Percent  action  initiated  by  time  ordered. 

X 

X 

X 

5 

Mean  Dissemination  Time. 

X 

X 

X 

8 

Percent  orders  clarification  requested. 

X 

X 

X 

11 

Percent  planning  time  forwarded. 

X 

X 

X 

7 

Time  from  mission  to  order. 

X 

X 

X 

6 

Time  to  decision  ratio. 

X 

X 

X 

4 

Number  of  orders  issued. 

X 

X 

13 

Percent  of  personnel  informed. 

X 

X 

X 

9 

Warning  to  operations  order  ratio. 

X 

X 

X 

X 

10 

Ability  to  develop  courses  of  action. 

X 

X 

X 

3 

Tactical  picture  quality. 

X 

X 

X 

1 

Display  processing  time. 

X 

12 

Table  3.  Facilitate  Commander's  Influence  of  Events 

The  MOEs  with  the  highest  performance  rankings  in  Table  3  are  "Tactical 
Picture  Quality"  and  "System  Contribution  to  the  Commander's  Perception  of  the  Enemy.” 
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This  result  is  consistent  with  fundamental  concepts  of  C^I  for  the  Warrior  and  maneuver 
warfare,  which  are  "to  provide  the  knowledge  and  understanding  the  commander  needs" 
so  that  his  decision  cycle  is  faster  than  his  opponent's  [Ref  2].  Measures  with  two  of  the 
three  highest  performance  rankings  in  Table  3  also  deal  with  the  decision  process  ("System 
Contribution  to  Conunander's  Perception  of  the  Enemy"  and  "Ability  to  Develop  Courses 
of  Action"),  while  percentage  ("Percent  Orders  Clarification  Requested")  and  event 
measures  ("Number  of  Orders  Issued")  have  lower  ranks.  Additionally,  the  two  measures 
with  the  lowest  performance  rankings,  "Display  Processing  Time"  and  "Number  of  Orders 
issued,"  can  be  used  to  measure  less  than  the  table  average  of  three  supporting  elements 
(as  measured  by  the  number  of  X's  along  a  row).  Doctrinal  requirement  (i),  dealing  with 
providing  the  commander  the  ability  to  view  the  battlefield  beyond  current  operations,  is 
weakly  supported  with  only  one  MOE  (one  X  in  the  (i)  column). 

2.  Doctrinal  Category:  Adapt  to  the  Situation 

"The  commander  must  be  free  to  move  and  lead  fi’om  any  point  in  the  battlespace. 
Therefore,  the  C^  system  must  have  the  flexibility  to  adapt  to  his  needs  rapidly."  [Ref  24] 

a.  Supporting  Elements 

(a)  A  C^  system  possess  the  characteristic  of  "flexibility."  [Ref  24] 

(b)  A  C^  system  should  "communicate  those  plans  and  orders  (see  category  1 
"translate  commander's  decisions  into  plans  and  orders")  to  subordinates."  [Ref  26] 

(c)  A  C^  system  "must  be  easy  to  establish,  easy  to  maintain,  and  reconfigure." 

[Ref  1] 
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(d)  The  system  "must  adapt  to  rapidly  changing  situations  and  exploit  fleeting 
opportunities."  [Ref.  27] 

(e)  Should  possess  the  "capability  to  support  merging  forces  and  expanding  to 
larger  MAGTFs  without  disrupting  continuity  of  operations."  [Ref  1] 

(f)  Should  possess  the  "capability  for  the  commander  to  view  friendly  and 
enemy  force  data  as  it  is  received."  [Ref  1] 

(g)  Should  possess  the  "capability  to  support  planning  on  the  move."  [Ref  1] 

(h)  "Must  satisfy  the  command  and  control  requirements  of  the  MAGTF 
commander  afloat,  ashore,  or  in  transition."  [Ref  1] 
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Measure  of  Effectiveness 


Time  from  mission  to  order. 


Time  to  decision  ratio. 

Set  up  and  tear  down  time. 


Percent  of  communications  with  alternate  routes. 
Grade  of  service. 


Measures  These  Supporting 
Elements 

X  X 


Performance 


System  access. 

■BB 

X 

5 

Display  processing  time. 

■n 

X 

12 

Database  size  consistency. 

Bl 

X 

11 

13 

BHin 


Tactical  picture  consistency. 


Mean  dissemination  time. 


Percent  informed. 
Ability  to  reconfigure. 


Percent  messages  displayed  inaccurately. 
Number  of  orders  issued. 


Number  of  duplicate  reports. 

Tactical  picture  quality. 

Table  4.  Adapt  to  the  Situation. 


For  the  measures  presented  in  Table  4,  number  counts  ("Number  of  Duplicate 
Reports"  and  "Number  of  Orders  Issued")  seem  to  be  ranked  lower  than  time  measures 
("Time  from  Mission  to  Order"  and  "Time  to  Decision  Ratio"),  which  reflects  the 
importance  of  tempo  in  maneuver  warfare,  especially  when  dealing  with  the  commander's 
ability  to  adapt  to  a  changing  situation.  Measures  dealing  with  displays  and  databases  also 
seem  to  be  less  important  than  decisionmaking  and  mobility  MOEs.  The  average  number 
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of  supporting  elements  that  can  be  measured  by  an  MOE  (the  average  number  of  X's  in  a 
row)  is  between  one  and  two. 

3.  Doctrinal  Category:  Support  Information  Requirements 
"The  commander's  support  system  must  provide  a  reliable,  secure,  fast  and 
durable  information  network  that  allows  him  to  implement  his  operational  concepts, 
information  must  be;  timely,  accurate,  complete,  objective,  useable,  relevant,  positioned 
properly,  mobile,  accessible  and  fijsed."  [Ref  24] 

a.  Supporting  Elements 

(a)  "There  must  be  a  means  to  provide  accurate  navigation  down  to  the  smallest 
unit  to  reduce  the  amount  of  communications  and  improve  command  and  control 
effectiveness."  [Ref  1] 

(b)  A  system  "provides  universal  availability  of  data  with  selected  access 
based  on  a  commander's  mission  and  needs."  [Ref  8] 

(c)  A  system  "provides  battle-relevant,  theater  and  national  databases  having 
common  data  elements."  [Ref  8] 

(d)  A  system  should  "facilitate  (the)  rapid,  distributed  and  unconstrained 
flow  of  information  in  all  directions."  [Ref  28] 

(e)  A  system  should  "filter,  fuze  and  prioritize  information."  [Ref  28] 

(f)  A  system  should  "provide  echelon  jumping  capability."  [Ref  28] 

(g)  A  system  should  "provide  vertical,  lateral,  and  diagonal  redundancy." 

[Ref  28] 
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(h)  A  system  should  "provide  the  commander  accurate,  timely  information 
and  ideas  for  developing  feasible  courses  of  action  and  making  logical  decisions."  [Ref 


26] 


(i)  "Satisfy  the  commander's  critical  information  requirements,  and  make  those 
elements  of  information  available  in  a  timely  fashion  and  in  the  most  useable  form, 
supporting  both  supply-push  and  demand-pull."  [Ref  1] 
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Measure  of  Effectiveness 

Measures  These  Supporting 

Elements 

Performance 

(a) 

(b) 

(C) 

(d) 

(e) 

(f) 

(g) 

(h) 

(i) 

Number  of  options  remaining. 

X 

14 

Time  to  decision  ratio. 

X 

9 

Percent  commander's  critical  information 

requirements  met. 

X 

X 

X 

X 

1 

Proportion  friendly  elements  engaged  (fratricide). 

X 

X 

13 

Grade  of  service. 

X 

8 

Ability  to  develop  courses  of  action. 

X 

11 

Display  processing  time. 

X 

X 

. 

16 

Database  size  consistency. 

B 

■ 

■ 

17 

Tactical  picture  consistency. 

X 

B 

■ 

■ 

7 

Digital  map  capability. 

X 

■ 

B 

6 

Ability  to  support  overlays. 

■ 

B 

10 

System  tasking  timeliness. 

X 

B 

3 

Percent  of  messages  displayed  inaccurately. 

■ 

B 

fl 

12 

Average  time  from  threat  detection  to  identification. 

■ 

■ 

■ 

fl 

X 

4 

Time  potential  threat  undetected  ratio. 

X 

X 

5 

Tactical  picture  quality. 

X 

X 

2 

Sensor  effectiveness. 

X 

15 

Table  5.  Support  Information  Requirements. 


For  Table  5,  the  measures  which  gauge  the  satisfaction  of  a  commander's 
information  requirements  ("Percent  Commander's  Critical  Information  Requirements 
Met"),  overall  system  speed  ("System  Tasking  Timeliness"),  and  knowledge  of  the  enemy 
("Tactical  Picture  Quality"  and  "Average  Time  from  Threat  Detection  to  Identification") 
rank  the  highest  in  measuring  overall  information  requirements.  Measures  of  subsystem 
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equipment  performance  (including  "Sensor  Effectiveness"  and  "Display  Processing 
Time"),  are  ranked  lowest  for  measuring  support  of  information  requirements.  Supporting 
elements  (f)  and  (g),  dealing  with  echelon  jumping  and  system  redundancy,  are  weakly 
supported  with  only  one  MOE. 

4.  Doctrinal  Category:  Exploit  MAGTF  Capabilities 
"The  support  system  must; 

-Support  the  MAGTF  during  all  types  of  deployment  and  employment 

-Enable  the  MAGTF  to  maintain  mobility  (note:  this  is  not  the  same  as  commander's 

freedom  to  move  in  the  battlespace). 

-Survive  enemy  weapons  systems  and  warfare 

-Sustain  the  MAGTF 

-Provide  the  flexibility  to  task  organize 

-Interface  with  other  systems  (Joint,  combined,  NCA,  other  services/federal 
agencies)  [Ref  24] 

a.  Supporting  Elements 

(a)  A  system  should  possess  the  characteristics  of  deployability,  mobility, 
flexibility,  integratability,  interoperability  [Ref  24], 

(b)  Should  "allow  seamless  transfer  of  information  within  naval  forces  and 
between  naval  forces  and  joint  commands."  [Ref  8] 

(c)  "Provides  rapid  simultaneous  access  of  multiple  users  throughout  the  chain 
of  command  and  to  external  commands  and  agencies  as  appropriate."  [Ref  8] 

(d)  Communications  must  be  "easy  to  establish,  maintain  and  reconfigure."  [Ref 


1] 


(e)  "C^  employment  options  cannot  be  considered  independent  of  deployment 
options."  [Ref  1] 
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(f)  A  system  should  have  the  capability  to  "deploy  rapidly,  operate  under 
austere  conditions  and  at  extended  ranges,  and  project  and  maintain  from  a  sea-base." 
[Ref  1] 

(g)  A  system  "should  be  able  to  withstand  disruptions  of  all  kinds  -  created 
by  the  enemy  or  self-induced."  [Ref  1] 


Measure  of  Effectiveness 

Measures  These 

Supporting  Elements 

Performance 

(a) 

(b) 

(C) 

(d) 

(a) 

(f) 

(g) 

Percent  communications  links  with  alternate  routes. 

X 

X 

X 

10 

Balance  of  traffic  throughout  network. 

X 

X 

12 

Satcom  supportability. 

X 

X 

B 

B 

D 

6 

Percent  Informed. 

B 

B 

B 

15 

Ratio  of  supplies  consumed  versus  provided. 

X 

B 

□ 

14 

Mean  time  response  to  jamming. 

B 

■ 

13 

Personnel  requirements  improvement. 

X 

B 

Q 

3 

Weapons  systems  equivalents. 

X 

B 

B 

11 

Database  size  consistency. 

B 

16 

Tactical  picture  consistency. 

■ 

B 

B 

1 

Transportation  requirements  improvement. 

D 

B 

B 

8 

Index  of  availability. 

■ 

B 

5 

Index  of  survivability. 

■ 

B 

X 

7 

Transportability  index. 

B 

B 

1 

4 

System  access. 

B 

B 

Bl 

1 

B 

9 

Ability  to  reconfigure 

□ 

□ 

X 

2 

Table  6.  Exploit  MAGTF  Capabilities 
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For  Table  6,  those  measures  which  deal  with  mobility  and  deployability  ("Ability  to 
Reconfigure"  and  "Personnel  Requirements  Improvement")  are  ranked  high  for  their 
performance  in  assessing  a  O'*!  system's  ability  to  exploit  MAGTF  capabilities. 
Interestingly,  measures  which  deal  with  combat  service  support  ("Ratio  of  Supplies 
Consumed  Versus  Provided")  and  information  ("Percent  Informed"  and  "Database  Size 
Consistency")  are  ranked  at  the  bottom.  All  supporting  elements  in  Table  4  can  be 
measured  by  at  least  three  MOEs. 

5.  Doctrinal  Category;  Support  CP  Effectiveness 

A  command  post  (C^  system)  needs  to  be  able  to  survive,  and  also  to  fight  [Ref  24], 
A  tradeoff,  however,  exists  between  fighting  and  survival  as  shown  in  Figure  6. 
Characteristics  that  help  a  C^  survive  may  impede  its  ability  to  fight,  and  vice-versa. 


Fieht 

Op  sec  Tradeoff 

Speed 

Mobility 

Simplicity 

Austerity 

CP  Design 

Signature 

Information 

Location 

Automation 

Deception 

Communication 

Dispersion 

Standardization 

Redundancy 

Qualified  Personnel 

Cover  and  Concealment 

Continuous  Operation 

Figure  6.  Survive  and  Fight  Tradeoff.  From  Ref  [24]. 
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(L  Supporting  Elements 

(a)  A  system  should  possess  the  characteristics  of  survivability  and 
sustainability.  [Ref  24] 

(b)  "Provides  rapid  simultaneous  access  of  multiple  users  throughout  the  chain 
of  command  and  to  external  commands  and  agencies  as  appropriate."  [Ref  8] 

(c)  "Displays  a  scaleable,  near  real-time,  shared  picture  of  the  battlefield."  [Ref 

8] 

(d)  "C^  facilities  must  be  able  to  operate  on  the  move."  [Ref  1] 

(e)  "There  is  a  need  to  reduce  the  number  of  transmissions  and  transmission 
time,  thereby  reducing  the  electronic  signature."  [Ref  1] 

(f)  Should  facilitate  "rapid  and  frequent  displacements."  [Ref  1] 

(g)  Organization  should  "ensure  unity  of  command."  [Ref  27] 

(h)  "Organization  should  also  ensure  reasonable  span  of  control  (3-7  units)." 

[Ref  26] 

(i)  Must  have  an  "echelonment  capability"  (A  and  B  commands  for 
displacement).  [Ref  1]. 
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Measures  of  Effectiveness 


Measures  These  Supporting  Performance 
Elements 


Communications  performance  index  (capacity,  XXXXXXXXX 
range,  security,  mobility,  dependability). 


Time  to  decision  ratio. 


Average  message  backlog. 


Percent  of  net  capacity  utilization. 


Grade  of  service. 

Percent  of  actions  initiated  by  time  ordered. 


Display  processing  time. 


Percent  of  personnel  informed. 


Set-up  and  tear-down  time. 
Ability  to  reconfigure. 


Personnel  requirements  improvement. 
System  tasking  timeliness. 


Percent  radio  links  up. 


Messages  lost  due  to  radio  links  down. 


Number  of  duplicate  reports. 
Tactical  picture  quality. 


Tactical  picture  consistency. 
Average  message  duration. 


Transportability  index. 


Index  of  survivability. 


Index  of  availability. 


Table  7.  Support  CP  Effectiveness 


In  the  category  of  Supporting  CP  Effectiveness,  MOEs  dealing  with 


communications  ("Communications  Performance  Index"),  system  displays  and  information 
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("Tactical  Picture  Quality"),  and  mobility  ("Transportability  Index")  are  ranked  high  in 
performance.  Specific  counts  and  percentage  measures  ("Percent  of  Personnel  Informed" 
and  "Number  of  Duplicate  Reports")  are  assessed  as  relatively  poor  measures  of  CP 
Effectiveness  throughout  the  table. 
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IV.  CONCLUSIONS 


A.  SUMMARY 

1.  Background 

Riding  the  crest  of  the  "information  age"  and  what  has  been  described  as  a 
"revolution  in  military  affairs,"  technologies  are  emerging  as  the  force  multipliers  of 
future  battlefields.  Current  initiatives  strive  to  push  information  and  intelligence  to 
increasingly  lower  echelons  of  command,  without  exceeding  the  commander's  capacity  to 
sift  and  assimilate  what  is  pertinent.  However,  due  to  past  shortcomings  in  C*I 
procurement,  and  the  high  cost  of  emerging  technologies,  upgrades  to  current  C'*!  systems 
and  new  C"*!  acquisitions  are  subjected  to  intense  analysis. 

2.  C'*I  Evaluation  Methodologies 

A  multitude  of  C'*!  evaluation  methodologies  exist,  each  approaching  the  problem  of 
comparative  analysis  in  a  slightly  different  manner.  Available  techniques  include  the 
COEA,  MCES,  MO  A,  HEAT,  evolutionary  upgrade  paths,  and  functional  decomposition 
methods.  Importantly,  C^I  evaluation  methodologies  can  be  combined  into  hybrid 
techniques  that  benefit  from  the  strengths  of  individual  techniques. 

Current  practice  in  the  Marine  Corps  and  other  services  has  established  the  COEA 
as  the  de  facto  framework  for  conducting  0^*1  system  comparative  analyses.  The  COEA 
methodology,  however,  is  broad  enough  to  accept  input  from  other,  potentially  more 
precise,  evaluation  techniques.  The  strength  of  the  MCES  lies  in  its  initial  focus  on 
problem  definition  and  the  environment,  rather  than  a  premature  focus  on  a  quantitative 
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model.  MCES  also  enjoys  a  commonality  of  terms  and  structure  which  simplify  its 
application.  The  MOA  focuses  on  obtaining  required  input  from  political  and  military 
decisionmakers  prior  to  defining  an  analytical  question.  HEAT  has  been  successfully 
applied  to  military  C^'l  systems,  and  represents  a  systematic  methodology  using 
pre-defined  measures.  Evolutionary  upgrade  path  concepts  help  provide  greater 
resolution  when  comparing  alternate  systems  with  similar  capabilities.  Functional 
decompositions  use  varied  approaches  which  couch  an  analysis  in  terms  of  required 
system  capabilities. 

Although  any  hybrid  combination  of  methodologies  would  only  be  developed  in  the 
context  of  the  analysis  at  hand,  one  such  combination  that  suggests  itself  might  be  a 
COEA  which  employs  the  first  four  modules  of  MCES  for  problem  definition,  and 
evolutionary  upgrade  paths  for  distinguishing  the  best  of  a  subset  of  "good"  systems. 

3.  MOEs 

The  identification  of  realistic  MOEs  is  a  common  thread  which  binds  the  various  O'*! 
evaluation  methodologies.  Unique  MOEs  may  be  required  for  a  specific  analysis,  and  can 
be  generated  using  MCES,  MOA,  evolutionary  upgrade  paths,  or  functional 
decompositions.  Importantly,  many  useful  MOEs  already  exist  from  past  analyses,  C”*! 
literature,  and  academia.  The  ACCES,  for  instance,  uses  a  pre-defined  set  of  measures 
which  remain  constant  from  analysis  to  analysis.  The  proposed  framework  for 
categorizing  MOEs  in  Chapter  III,  based  on  relevant  doctrine,  provides  a  tool  for 
identifying  and  maintaining  those  MOEs  with  broad  applicability  to  Marine  Corps  C''! 
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system  evaluation.  The  framework  can  change  dynamically  with  emerging  doctrine,  and 
can  also  incorporate  new  MOEs  developed  by  the  application  of  MCES  or  other  MOE 
generating  methodologies. 

Importantly,  the  framework  provides  a  starting  point  for  increased  resolution  in 
categorizing  C''I  MOEs  for  use  in  Marine  Corps  related  analysis.  Potential  sub-categories 
that  may  provide  greater  detail  include  functionality  (i.e.,  is  a  component  primarily  for  C^, 
communications,  intelligence,  or  interoperability?),  and  compatibility  (e.g.,  is  the  system 
employable  inside  existing  vehicle  spaces  and  CP  configurations?). 

For  the  measures  identified  in  the  literature,  those  MOEs  which  go  beyond  number 
counts  and  percentages  seem  to  promise  better  performance  in  measuring  doctrinal  0“*! 
system  requirements.  Higher  level  measures,  especially  Tactical  Picture  Quality,  enjoyed 
high  performance  rankings  across  multiple  doctrinal  categories.  The  identification  of 
existing  measures  which  provide  high  performance  provides  a  logical  starting  point  for 
refinement  and  identification  of  similarly  powerful  MOEs. 
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APPENDIX  A.  MOE  DESCRIPTIONS 


A.  DEFINITIONS  AND  RATIONALE 

1.  MOE:  System  Contribution  to  Commander's  Perception  of  the  Enemy 

Definition:  The  total  number  of  enemy  units  upon  which  there  is  intelligence 

information  in  a  database,  the  age  of  the  intelligence  information,  and  current  time  within 
the  scenario.  The  output  is  a  cumulative  graph  of  the  number  of  enemy  units  upon  which 
there  is  intelligence  information  versus  time.  [Ref  23] 

Rationale:  The  commander's  perception  of  the  battlefield  has  a  direct  impact  on  the 
decision  that  he  will  make.  This  measure  attempts  to  assess  the  contribution  of  the  Cl 
system  to  the  commander's  decision  making  process,  and  provide  for  a  comparative 
analysis  of  alternatives  for  supporting  the  commander's  decision  making  process. 

Supports:  Facilitate  Commander's  Influence  of  Events. 

2.  MOE:  Percent  Action  Initiated  by  Time  Ordered 

Definition:  The  percentage  of  all  actions  initiated  in  response  to  orders  that  are 
initiated  within  the  time  specified  by  the  order  [Ref  23]. 


Percent  Actions  Initiated  on  Time  = 


Number  of  actions  initiated  by  time  ordered 
Number  of  actions  ordered 


Rationale:  This  measure  addresses  the  timeliness  of  subordinate  reaction  to  orders, 
and  by  extension  the  timeliness  of  the  C^I  system  in  processing,  transmitting,  and 
displaying  the  orders  to  subordinate  commander's. 

Supports:  Facilitate  Commander's  Influence  of  Events,  Support  CP  Effectiveness. 


3.  MOE:  Mean  Dissemination  Time 


Definition:  The  time  required  to  disseminate  an  order,  directive,  or  warning  to  all 
elements  at  the  next  lower  echelon  of  command  [Ref  23], 

\  r  r\-  j  •  T-  2  (average  dissemenation  time  for  each  order') 

Mean  Dissemenation  Time  =  — ^ ^ - : — :: — - - 

number  of  orders 

Rationale:  This  measure  directly  measures  the  timelin  ss  of  a  C^I  system  in 
disseminating  orders. 

Supports:  Facilitates  Commander's  Influence  of  Events,  Adapt  to  the  Situation. 

4.  MOE:  Percent  Orders  Clariflcation  Requested 

Definition:  The  percentage  of  total  orders  issued  (can  include  fi'ag  orders),  for 
which  any  subordinate  element  requested  clarification  [Ref  29]. 

Percent  Orders  Clarification  Requested  =  ^fT>cauoa  ^ 

Rationale:  This  measure  addresses  the  clarity  of  the  orders  issued,  indirectly 
addressing  the  quality  of  orders  produced  by  the  system.  It  is  also  an  indirect  measure 
of  the  like-mindedness  of  subordinate  and  senior  commanders. 

Supports:  Facilitates  Commander's  Influence  of  Events. 

5.  MOE:  Time  to  Decision  Ratio 

Definition:  The  proportion  of  time  from  receipt  of  mission  until  time  of  execution 
that  is  devoted  to  a  commander's  decision  [Ref  23]. 
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Time  to  Decision  Ratio  = 


{time  order  approved)  -  {time  receipt  mission) 
{time  execution  ordered)  -  {time  receipt  mission) 


Rationale:  This  measure  addresses  the  effectiveness  of  a  Cl  system.  It  is  assumed 
that  the  Cl  system  which  provides  the  most  relevant  information,  the  most  assistance  in 
preparing  an  order,  the  best  standard  operating  procedures,  and  the  best  human  interface 
with  technology  will  require  less  time  to  finalize  an  order. 

Supports:  Facilitates  Commander's  Influence  of  Events,  Adapt  to  the  Situation, 
Support  Information  Requirements,  Support  CP  Effectiveness. 

6.  MOE:  Percent  Planning  Time  Forwarded 

Definition:  The  percentage  of  total  planning  time  available  that  an  echelon  allows  to 
all  lower  echelons  [Ref  23]. 


Percent  Planning  Time  Forwarded  =  (1  -  Time  to  Decision  Ratio)  X 100 

Rationale:  This  measure  addresses  Cl  system  effectiveness  by  assessing  the 
planning  time  required  to  develop  and  issue  an  order  in  relation  to  the  time  available. 
Marine  Corps  doctrine  calls  for  passing  fifty  percent  of  the  available  time  to  mission  to 
subordinates. 

Supports:  Facilitate  Commander's  Influence  of  Events. 
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7.  MOE:  Time  from  Mission  to  Order 


Definition:  The  elapsed  time  at  one  echelon  of  command  from  the  moment  of 
receiving  a  mission  to  the  moment  of  issuing  the  responsive  order  to  the  next  lower 
echelon  [Ref  23], 

Rationale:  This  measure  directly  measures  the  timeliness  of  the  command  fianction. 
It  includes  planning  time,  decision  time,  and  time  to  prepare  and  disseminate  the  order.  It 
is  assumed  that  the  best  C*I  system  will  have  the  fastest  time  from  mission  to  order.  The 
measure  may  also  be  used  to  compare  the  command  function  in  different  environments, 
e.g.,  mobile  vs  stationary,  or  afloat  vs  ashore. 

Supports:  Facilitate  Commander's  Influence  of  Events,  Adapt  to  the  Situation. 

8.  MOE:  Number  of  Orders  Issued 

Definition.  A  simple  number  count  of  the  orders  issued  for  a  given  operation  [Ref. 
29]. 

Rationale:  This  measure  directly  measures  the  amount  of  command  and  control  and 
is  considered  an  indication  of  amount  needed,  which  itself  relates  to  cost  or  burden. 
Additionally,  this  assesses  indirectly  the  commander's  perception  of  the  battlefield  (better 
perception  relates  to  less  orders),  and  the  extent  to  which  subordinate  commanders  are 
receiving  ideas  and  concepts  (the  better  they  understand  the  situation  the  less  direction 
they  will  require). 

Supports:  Facilitate  Commander's  Influence  of  Events,  Adapt  to  the  Situation. 
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9.  MOE:  Percent  of  Personnel  Informed 

Definition:  The  percentage  of  personnel  aware  of  an  information  item  when  asked 
[Ref  29], 


number  of  personnel  aware  of  item  ^ 
number  of  personnel  asked 

Rationale:  This  measure  addresses  the  extent  to  which  a  system  provides  the  same 
information  to  all  users,  or  the  extent  to  which  subordinate  commanders  receive  ideas  and 
information,  or  how  well  a  system  is  performing  in  different  environments  (e  g.,  on  the 
move  vs  stationary). 

Supports:  Facilitate  Commander's  Influence  of  Events,  Adapt  to  the  Situation, 
Exploit  MAGTF  Capabilities,  Support  CP  Effectiveness. 

10.  MOE:  Warning  Order  to  Operations  Order  Ratio 

Definition:  The  number  of  warning  orders  divided  by  the  number  of  operations 
orders  (can  include  frag  orders)  [Ref  29], 

Rationale:  This  measure  assesses  the  extent  to  which  subordinate  commanders  are 
receiving  ideas  and  concepts,  vice  data.  It  also  assesses  the  extent  to  which  all 
commanders  are  operating  off  the  same  tactical  picture. 

Supports:  Facilitate  Commander's  Influence  of  Events. 

11.  MOE:  Ability  to  Develop  Courses  of  Action 

Definition:  The  time  necessary  to  develop  courses  of  action  given  commander’s 
guidance  [Ref  30]. 
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Rationale:  This  measure  assesses  the  extent  to  which  a  Cl  system  facilitates  the 
development  of  COAs.  It  is  assumed  a  superior  O'*!  system  will  be  faster  than  an  inferior 
system. 

Supports:  Facilitate  Commander's  Influence  of  Events,  Support  Information 

Requirements. 

12.  MOE:  Tactical  Picture  Quality 

Definition:  Measured  by  a  function  of  completeness  of  tactical  picture,  accuracy  of 
identification,  and  accuracy  of  position  indication  [Ref  31]. 

Rationale:  This  measure  assesses  the  accuracy  of  the  tactical  information  provided 
by  the  Cl  system.  Tactical  picture  quality  will  impact  the  commander's  perception  of  the 
battlefield,  as  well  as  staff  planning. 

Supports:  Facilitate  Commander's  Influence  of  Events,  Adapt  to  the  Situation, 
support  Information  Requirements,  Support  CP  Effectiveness. 

13.  MOE:  Display  Processing  Time 

Definition:  Duration  of  time  between  receipt  of  information  at  C'*!  node  and  time 
when  information  first  appears  at  a  display  where  decision-making  process  begins  [Ref 
31]. 

Rationale:  This  measure  assesses  the  timeliness  of  a  portion  of  the  C''!  system.  It  is 
assumed  that  the  system  with  the  fastest  display  processing  time  will  be  superior  to  those 
with  slower  display  processing  times.  Note  that  this  measure  corresponds  to  the  observe 
portion  of  the  Boyd  Cycle. 
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Supports:  Facilitate  Commander's  Influence  of  Events,  Adapt  to  the  Situation, 


support  Information  Requirements,  Support  CP  Effectiveness. 

14.  MOE:  Set-up  and  Tear-down  Time 

Definition:  Set-up  time  is  the  elapsed  time  from  CP  site  selection  to  operational 
capability  of  the  C'*!  system.  Tear-down  time  is  the  time  required  to  dismantle  the  system 
and  load  it  aboard  transportation  [Ref  30]. 

Rationale:  This  measure  addresses  the  flexibility  of  a  C*!  system.  A  system  which 
can  be  installed  and  dismantled  quickly  will  be  more  responsive,  survivable,  and  mobile. 

Supports:  Adapt  to  the  Situation,  Support  CP  Effectiveness. 

15.  MOE:  Percent  Communications  with  Alternate  Routes 

Definition:  The  percentage  of  all  established  node  to  node  communications  links 
that  also  have  an  existing  alternate  route  for  communications  [Ref  29], 

Rationale:  This  measure  addresses  the  robustness  of  a  CT  system.  Alternate 
communications  routes  ease  message  backlogs,  resist  jamming  effects,  and  increase 
timeliness. 

Supports:  Adapt  to  the  Situation,  Exploit  MAGTF  Capabilities. 

16.  MOE:  Grade  of  Service 

Definition:  The  probability  that  a  subscriber  at  any  randomly  chosen  instant  will  be 
able  to  obtain  a  circuit  connection  to  his  party  [Ref  29], 

Rationale:  This  measure  assesses  the  responsiveness  and  quality  of  a  C**!  system.  A 
better  grade  of  service  suggests  a  better  flow  of  information  between  echelons  of 
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command  and  within  CPs.  Comparing  grades  of  service  allows  analysis  of  CPs  in  different 
states  (e  g.,  mobile  vs  stationary). 

Supports:  Adapt  to  the  Situation,  Support  Information  Requirements,  Support  CP 
Effectiveness. 

17.  MOE:  System  Access 

Definition:  Proportion  of  time  requested  that  the  MAGTF  commander  was  given 
access  to  a  specific  (shipboard)  system  to  support  (OTH)  operations  [Ref  3 1]. 

Rationale:  This  measure  assesses  the  responsiveness  and  supportability  of  a  C''! 
system.  Due  to  equipment  constraints  afloat,  as  well  as  satellite  access  and  bandwidth 
constraints  afloat  or  ashore,  the  MAGTF  commander  may  not  enjoy  unlimited  access  to 
C''!  assets.  Examples  include  limited  W-3  shipboard  satellite  antennas,  and  limited  SHF 
satcom  bandwidth  for  GMF  terminals  ashore. 

Supports:  Adapt  to  the  Situation,  Exploit  MAGTF  Capabilities. 

18.  MOE:  Database  Size  Consistency 

Definition:  Proportion  of  database  consistency  between  the  MAGTF  headquarters 
and  selected  subordinate  units  [Ref  31]. 

Database  size  consistency  =  1  -  - 

where:  x  =  database  size  of  MAGTF  headquarters 
yn  =  database  size  of  nth  selected  subordinate  unit 
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Rationale:  This  measure  addresses  the  consistency  of  a  system,  and  the  extent 
to  which  commanders  and  staffs  at  different  echelons  have  access  to  the  same  amount  of 
information. 


Supports:  Adapt  to  the  Situation,  Support  Information  Requirements,  Exploit 
MAGTF  Capabilities. 

19.  MOE:  Tactical  Picture  Consistency 

Definition:  Measured  by  relative  comparisons  of  displays,  similarity  of 

identification,  and  similarity  of  position  information  [Ref  31]. 

Rationale:  This  measure  assesses  the  extent  to  which  commanders  and  staffs  at 

different  echelons  are  operating  with  the  same  tactical  picture.  It  may  be  correlated  with 
number  of  orders  issued,  database  size  consistency  and  percent  orders  correction 
requested. 

Supports:  Adapt  to  the  Situation,  Support  Information  Requirements,  Exploit 
MAGTF  Capabilities,  Support  CP  Effectiveness. 

20.  MOE:  Ability  to  Reconflgure 

Definition:  Measured  by  time  to  initialize  system,  time  to  backup  system,  and  time 
to  reboot  system  [Ref  30]. 

Rationale:  This  system  addresses  CT  system  flexibility  and  mobility.  It  is  assumed 
that  a  quicker  time  for  each  element  of  the  measure  is  desirable. 

Supports:  Adapt  to  the  Situation,  Exploit  MAGTF  Capabilities,  Support  CP 


Effectiveness. 


21.  MOE:  Percent  Messages  Displayed  Inaccurately 

Definition:  The  percentage  of  messages  received  that  were  displayed  inaccurately 
[Ref  29], 

Rationale:  This  measure  assesses  C*l  system  accuracy  and  system  information 
reliability. 

Supports:  Adapt  to  the  Situation,  Support  Information  Requirements. 

22.  MOE:  Number  of  Duplicate  Reports 

Definition:  A  number  count  of  the  number  of  individual  reports  displayed  multiple 
times  [Ref  31]. 

Rationale:  This  measure  addresses  the  quality  of  a  O'*!  system.  It  also  forms  a  basis 
for  comparison  between  CPs  in  diflFerent  states  (e.g.,  does  the  mobile  CP  have  more 
duplicate  reports  than  the  stationary  CP. 

Supports:  Adapt  to  the  Situation,  Support  CP  Effectiveness. 

23.  MOE:  Number  of  Options  Remaining 

Definition:  The  number  of  decision  points  open  multiplied  by  the  number  of  options 
for  each  decision  point  [Ref  23], 

Rationale:  This  measure  addresses  the  quality  of  a  system  in  supporting  planning  for 
courses  of  action,  and  the  amount  of  flexibility  left  to  the  commander.  The  C^*!  system 
which  presents  the  greater  number  of  tactical  options  for  a  commander  is  assumed  to  be 
superior. 

Supports:  Support  Information  Requirements. 
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24.  MOE:  Percent  Commander's  Critical  Information  Requirements  Met 

Definition:  The  percentage  of  commander's  critical  information  requirements  met, 
where  commander's  critical  information  requirements  are  defined  by  Marine  Corps 
doctrine  [Ref  30], 

Rationale:  This  measure  assesses  the  quality  of  a  system.  The  Cl  system  which 
presents  the  greatest  percentage  of  CCIRs  is  assumed  superior. 

Supports:  Support  Information  Requirements. 

25.  MOE:  Proportion  Friendly  Elements  Engaged  (Fratricide) 

Definition:  The  number  of  friendly  elements  erroneously  engaged  divided  by  the 
number  of  fiiendly  elements  in  the  engagement  area  [Ref  23]. 

Rationale:  This  measure  addresses  one  of  the  most  catastrophic  failures  in  Cl, 
"fiiendly  fire."  By  extension,  this  measure  assesses  the  quality  of  the  tactical  picture  at 
lower  echelons,  and  the  extent  to  which  all  echelons  are  operating  from  the  same  tactical 
picture. 

Supports:  Support  Information  Requirements. 

26.  MOE:  Digital  Map  Capability 

Definition:  Measured  by  time  to  generate  map  display  and  time  to  generate  scanned 
map  [Ref  30]. 

Rationale:  This  measure  addresses  the  timeliness  of  the  O'*!  system  with  regard  to 
map  displays. 

Supports:  Support  Information  Requirements. 
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27.  MOE:  Ability  to  Support  Overlays 

Definition:  Measured  by  time  to  generate  map  overlays  and  time  to  disseminate  map 
overlays  to  lower  echelons  other  staff  sections  [Ref  30], 

Rationale:  This  measure  addresses  the  timeliness  of  a  Cl  system  with  regard  to 
overlays. 

Supports:  Support  Information  Requirements. 

28.  MOE:  System  Tasking  Timeliness 

Definition:  The  time  information  is  received  from  a  database  inquiry  minus  the  time 
the  inquiry  was  initiated  [Ref  31]. 

Rationale:  This  measure  assesses  the  timeliness  of  a  Cl  system.  It  can  also  be  used 
to  compare  the  system  timeliness  in  different  configurations  (e  g.,  mobile  or  stationary). 
Supports:  Support  Information  Requirements,  Support  CP  Effectiveness. 

29.  MOE:  Average  Time  from  Threat  Detection  to  Identification 
Definition:  The  average  of  all  times  from  threat  detection  to  threat  identification 

[Ref  32]. 

Rationale:  This  measure  addresses  the  timeliness  of  the  intelligence  function  of  a 
C'*!  system. 

Supports:  Support  Information  Requirements. 
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30.  MOE:  Time  Potential  Threat  Undetected  Ratio 


Definition:  The  length  of  time  an  enemy  threat  is  within  the  system  sensor  envelope 
before  detection  divided  by  the  total  length  of  time  an  enemy  threat  is  within  the  envelope 
[Ref  32], 

Rationale:  This  measure  addresses  the  sensor  effectiveness  of  a  Cl  system.  By 
extension,  the  measure  can  be  used  to  assess  the  timeliness  of  reporting  within  the  system 
(i.e.,  time  before  detection  can  actually  be  time  before  threat  detection  report  is  displayed 
at  a  particular  echelon). 

Supports:  Support  Information  Requirements. 

31.  MOE:  Sensor  Effectiveness 

Definition:  The  number  of  reports  received  by  sensor  type  divided  by  the  total 
number  of  sensor  reports  received  [Ref  31]. 

Rationale:  This  measure  assesses  the  effectiveness  of  the  sensors  associated  with  a 
C''I  system. 

Supports:  Support  Information  Requirements. 

32.  MOE:  Balance  of  Traffic  Through  Network 

Definition:  The  average  utilization  of  each  communications  node  and  link  [Ref  29]. 

Rationale:  This  measure  assesses  the  criticality  of  communications  nodes  and  links, 
as  well  as  the  overall  system  survivability  and  compactness  of  the  system  (e  g.,  a  system 
with  low  utilization  of  many  links  may  in  fact  have  octraneous  parts). 

Supports:  Exploit  MAGTF  Capabilities. 
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33.  MOE:  SatCom  Supportability 

Definition:  The  theoretical  satellite  uplink  channel  capacity  is  measured  by: 


R=WSI 


where: 

R  =  Transmitted  data  rate  capacity  of  a  satellite  uplink  channel 
W  =  spread  spectrum  bandwidth 

S  =  uplink  signal  Effective  Instantaneous  Radiated  Power  (EIRP) 

E/N^  =  required  post  correlation  signal-to-noise 

y  =  signal  suppression  to  the  transponder 

J  =  enemy  jammer  EIRE 

I  =  sum  of  other  uplink  signals'  EIRP 

E  =  satellite  full  output  downlink  EIRP 

L  =  downlink  path  loss 

K  =  Boltzman's  constant 

G/T  =  receiver  Earth  terminal  figure  of  merit 


[Ref  29] 


Rationale:  This  measure  assesses  the  amount  of  bandwidth  available  to  a  C"*!  system 
utilizing  SHF  satcom  communications. 

Supports:  Exploit  MAGTF  Capabilities. 
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34.  MOE:  Ratio  of  Supplies  Consumed  Versus  Provided 

Definition:  The  ratio  of  the  quantity  of  supplies  (by  class)  consumed  to  the  quantity 
of  supplies  provided  by  the  CSS  system  [Ref  23]. 

Rationale:  This  measure  addresses  the  relative  degree  to  which  supply  consumption 
can  be  satisfied  by  the  CSS  system.  It  is  assumed  that  the  faster  C''!  system  will  speed  and 
enhance  the  satisfaction  of  supply  demands  thereby  reducing  the  ratio  toward  one. 

Supports:  Exploit  MAGTF  Capabilities. 

35.  MOE:  Mean  Time  Response  to  Jamming 

Definition:  The  arithmetic  mean  of  each  elapsed  response  time  to  jamming  of 
fiiendly  communications  [Ref  33]. 

Rationale:  This  measure  addresses  the  effectiveness  of  one  countermeasure  aspect 
of  a  C''I  system. 

Supports:  Exploit  MAGTF  Capabilities. 

36.  MOE:  Personnel  Requirements  Improvement 

Definition:  The  net  change  in  personnel  requirements,  including  operators  and 
maintainers,  of  one  O'*!  system  over  another  [Ref  34]. 

Rationale:  This  measure  assesses  the  personnel  overhead  and,  by  extension,  the 
complexity  of  a  C*I  system.  Results  can  also  be  linked  to  system  deployability. 

Supports:  Exploit  MAGTF  Capabilities,  Support  CP  Effectiveness. 
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37.  MOE:  Weapons  Systems  Equivalents 

Definition:  The  number  count  of  a  particular  weapons  system  which  can  be 
considered  equivalent  to  the  C'*!  system  in  question  (e.g.,  1  JTIDS  equipped  F-15  is 
equivalent  to  1.3  non- JTIDS  equipped  F-15s)  [Ref  35], 

Rationale:  This  measure  assesses  the  force  effectiveness  of  a  C'*!  system,  and  puts 
the  result  in  terms  easily  understandable  to  multiple  warfighting  specialties  (e.g.,  an  armor 
'icer  can  readily  visualize  the  value  of  a  C^I  system  if  he  knc  s  its  tank  battalion 
equivalence).  The  enumeration  of  this  measure  will  usually  require  a  separate  analysis. 

Supports:  Exploit  MAGTF  Capabilities. 

38.  MOE:  Transportation  Requirements  Improvement 

Definition:  The  net  change  in  transportation  requirements  of  one  C*I  system  over 
another  [Ref  34]. 

Rationale:  This  measure  addresses  C*I  system  deployability  and  support 

requirements.  It  may  also  provide  a  measure  of  tactical  mobility. 

Supports:  Exploit  MAGTF  Capabilities. 

39.  MOE:  Index  of  Availability 

Definition:  Consists  of  some  or  all  of  the  following  availability  measures; 

Operational  Availability  =  (MTBF)  /  (MTBF  +  MDT) 

Inherent  Availability  =  (MTBF)  /  (MTBF  +  MTTR) 

Achieved  Availability  =  (MTBF)  /  (MTBF  +  MTTR  +  MTTP) 
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where; 


MTBF  =  mean  time  between  failure 
MDT  -  mean  down  time 
MTTR  =  mean  time  to  repair 

MTTP  =  mean  time  to  perform  scheduled  maintenance 
[Ref.  32], 

Rationale:  This  measure  addresses  reliability  and  availability  aspects  of  a  C'*! 
system. 

Supports:  Exploit  MAGTF  Capabilities,  Support  CP  Effectiveness. 

40.  MOE:  Index  of  Survivability 
Definition:  Equals: 

(1-Grade  of  Service /Grade  of  Service 
[Ref  32]. 

Rationale:  This  measure  assesses  system  survivability  in  terms  of  communications 
disruption. 

Supports:  Exploit  MAGTF  Capabilities,  Support  CP  Effectiveness. 

41.  MOE:  Transportability  Index 
Definition: 


T.-  (m^)(d^/d^  ifd.>d^ 


73 


where. 


T.  =  Transportability  index 
m  =  total  jth  subsystem  volume 
d^.  =  average  density  of  jth  subsystem 
d^  =  maximum  load  density  for  a  fully  loaded  vehicle 
[Ref.  32]. 

Rationale:  This  measure  assesses  deployability,  mobility,  and  logistics  requirements 
of  a  Cl  system. 

Supports:  Exploit  MAGTF  Capabilities,  Support  CP  Effectiveness. 

42.  MOE:  Communications  Performance  Index 

Definition:  The  weighted  sum  of  the  ratios  of  observed  performance  to  required 
performance  over  a  set  of  requirements  [Ref  29]. 

CPI  =  W,  (P,  /R)  +  W,  (P,/RJ  +  ...  +  fV,  (PJRJ 
where: 

W.  =  relative  weight  of  each  requirement 
P.  =  performance  observed 
R.  =  required  performance 

Rationale:  This  measure  is  a  generic  assessment  of  a  C''I  system  over  a  number  of 
stated  requirements.  Requirements  could  include:  direct  communication  capacity,  organic 
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communication  equipment,  conference  call  capability,  specific  range,  security,  mobility, 
dependability,  vulnerability,  etc. 

Supports:  Support  CP  Effectiveness. 

43.  MOE:  Average  Message  Backlog 

Definition:  The  average  number  of  messages  awaiting  transmission  [Ref  29]. 

Rationale:  This  measure  addresses  the  timeliness  of  a  C'*!  system. 

Supports:  Support  CP  Effectiveness. 

44.  MOE:  Percentage  of  Net  Capacity  Utilization 

Definition:  The  total  time  a  communication  net  carries  traffic  divided  by  the  total 
time  a  net  is  observed  [Ref  29]. 

Rationale:  This  measure  can  be  used  to  address  the  necessity  of  a  circuit,  but  is 
more  often  used  to  determine  whether  circuits  approach  overloading. 

Supports:  Support  CP  Effectiveness. 

45.  MOE:  Percent  Radio  Links  Up 

Definition:  The  number  of  radio  links  operational  divided  by  the  total  number  of 
radio  links  [Ref  35], 

Rationale:  This  measure  addresses  the  performance  of  the  communications  aspects 
of  a  C‘*I  system,  and  can  be  used  to  compare  system  performance  in  different  locations  and 
states. 

Supports:  Support  CP  Effectiveness. 
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46.  MOE:  Number  of  Messages  Lost  Due  to  Radio  Links  Down 

Definition:  A  number  count  of  the  messages  that  were  not  transmitted  or  which  lost 
their  time-related  value  as  a  result  of  a  radio  circuit  being  down  [Ref  35], 

Rationale:  This  measure  addresses  the  reliability  of  information  flow  in  a  C"! 
system. 

Supports:  Support  CP  Effectiveness. 

47.  ]*  lOE:  Average  Message  Duration 

Definition:  The  sum  of  message  transmission  times  divided  by  the  number  of 
messages  [Ref  29]. 

Rationale:  This  measures  addresses  the  survivability  of  a  Cl  system  in  an  EW 
environment. 

Supports:  Support  CP  Effectiveness. 
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APPENDIX  B.  ANALYSIS  OF  QUESTIONNAIRE  DATA 


Questionnaires  were  completed  by  twelve  respondents,  including  eleven  Marine 
Officers  and  one  professor.  The  Marine  participants  included  two  infantry  officers  (both 
company  grade),  two  artillery  officers  (one  field  grade,  one  company  grade),  and  six 
communications  officers  (one  field  grade  and  five  company  grade),  who  came  from  either 
the  Joint  ,  the  Electrical  Engineering,  or  the  Operations  Research  curriculums.  The 
faculty  participant  was  an  Operations  Research  professor  with  extensive  expertise  in 
combat  modelling.  The  questionnaires  are  enclosed  at  the  end  of  this  appendix. 

Respondents  were  initially  asked  to  rank  the  MOEs  based  on  three  categories, 
including  cost,  ease  of  use,  and  performance.  In  practice,  however,  the  questionnaires 
proved  to  be  an  unsuitable  method  to  obtain  cost  and  ease  of  use  data,  since  both 
categories  are  most  often  a  function  of  the  C*!  system  being  analyzed.  Performance 
proved  to  be  a  tractable  category,  and  complete  ranked  lists  were  obtained  from  each  of 
the  12  participants. 

The  ranked  lists  were  analyzed  using  a  variant  of  the  Method  of  Successive  Interval 
Scaling,  where  experts  are  asked  to  rank  instances  in  terms  of  how  much  of  a  property 
they  possess  [Ref  25].  If  Xj  represents  an  expert's  interval  scaled  feelings  about  the 
amount  of  the  property  possessed  by  instance  j,  then  the  methodology  requires  two 
assumptions.  The  assumptions  are: 

(1)  over  an  infinite  number  of  experts,  Xj  is  distributed  normally  with  mean  Sj  and 
homogeneous  variance  o^,  for  all  j. 
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(2)  The  correlation  coefficient  between  Xj  and  X,  is  the  same  for  all  ij. 

In  applying  the  method,  a  frequency  matrix  of  fjj  values  was  obtained  by  counting  the 
number  of  times  the  MOE  in  column  j  was  ranked  above  the  MOE  in  row  i.  Table  8 
displays  the  matrix  resulting  from  the  first  doctrinal  bin. 


Table  8.  Frequency  Matrix 

Next,  all  the  values  in  the  f'j  matrix  are  converted  to  proportions  by  dividing  each  f'j 
value  by  f'j  +  as  shown  in  Table  9.  Note  that  the  diagonal  entries  are  generated  on  the 
premise  that  forjudges  who  did  not  know  they  were  ranking  an  instance  against  itself,  one 
would  expect  half  the  judges  to  go  one  way,  and  half  the  other  way.  . 
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Table  9.  Proportion  Matrix 


Using  the  normality  assumption,  each  of  the  proportions  may  be  mapped  into  a 
standard  normal  distribution  by  using  the  inverse  cumulative  distribution  function.  By 
convention,  proportions  greater  than  .98  or  less  than  .02  are  treated  as  gaps  in  the  matrix. 
Table  10  displays  the  resulting  z-  matrix. 
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A 

B 

C 

D 

E 

F 

G 

H 

1 

J 

K 

L 

M 

-0.6 

-0.91 

-1.34 

-0.91 

-0.6 

-0.6 

* 

-0.91 

-0.91 

-0.35 

0.35 

* 

0.6 

-0.35 

-1.34 

-0.11 

-0.11 

0.11 

-1.34 

-0.35 

-0.35 

-0.11 

0.91 

-1.34 

0.91 

0.35 

-0.6 

0.35 

0.11 

* 

-1.34 

-0.11 

-0.35 

0.6 

0.91 

-0.91 

1.33 

1.34 

0.6 

&  ■ 

0.6 

0.11 

* 

-0.6 

0.35 

0.35 

1.34 

* 

0.11 

0.91 

0.11 

-0.35 

-0.6 

u 

-0.11 

0.6 

-0.91 

-0.35 

-0.35 

0.6 

0.91 

-0.91 

0.6 

0.11 

-0.11 

-0.11 

0.11 

b 

0.11 

-0.91 

-0.35 

-0.11 

0.11 

0.6 

-0.91 

0.6 

-0.11 

* 

* 

-0  6 

-0  11 

q 

-1.34 

-0.91 

-0.91 

0.35 

0.91 

-0.91 

* 

1.34 

1.34 

0.6 

0.91 

0.91 

1.34 

0.91 

0.6 

1.34 

* 

-0.11 

0.91 

0.35 

0.11 

-0.35 

0.35 

0.35 

i  0.91 

-0.91 

0 

-0.35 

06 

0  91 

-0.6 

0.91 

0.35 

0.35 

-0.35 

0.35 

'  0.11 

0.91 

-0.6 

0.35 

s 

0.35 

0.6 

-0.11 

0.35 

0.11 

-0.6 

-1.34 

-0.6 

-0.11 

-0.35 

-1.34 

-0.6 

-0.35 

0  i 

0.6 

* 

-0.35 

-0.91 

-0.91 

* 

-0.91 

-0.6 

-0.91 

* 

-0.91 

-0.6 

-0.6 

■■  b  ' 

* 

* 

1.34 

0.91 

-0.11 

0.91 

0.91 

0.91 

0.11 

0.6 

0.11 

* 

* 

Table  10.  Matrix 


For  those  columns  without  holes  a  scale  value  for  the  MOE  listed  in  the  column 

header  can  be  generated  by  the  column  mean.  In  Table  10,  columns  B,E,F,I  and  J  are 

complete.  The  remaining  columns  may  be  solved  as  a  set  of  simultaneous  equations  of  the 
form,  njSj-'^Sj  =  '^Zij  y  =  1,2, ...«.  The  solutions  to  this  set  of  equations  will  be  a 

/  i 

scale  value  for  the  MOE,  estimating  the  column  mean.  For  the  problem  at  hand,  the 
following  set  of  equations  is  generated: 
lOS^  -  Sc  -  Sd  -  Sq  -  Sj,  -  Sl  =  6.7489 
1 1  Sc  -  -  Sh  -  -  Sl  -  =  .077965 

lOSo  -  S^  -  Sc -Sh-Sj,-S^  = -5.56989 
10Sc-S^-Sh-S^-S,-S^  =  2.99781 
IOSh  -  Sc  -  S„  -  Sc -S^-S^  =  -9.19499 
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llS^-S^-Se-Sp-SQ-SH-S,  =  4.19441 
9S,-S^-Se-S^- 8^  =  6.67091 
9S^  -  S,  -  So -S,-Sh  = -5.72169 

where  Sj  equals  the  scale  value  for  column  i.  In  this  instance,  the  resulting  interval 
scale  values  all  fall  between  -1  and  1,  and  may  be  linearly  transformed  into  a  more 
appropriate  scale.  In  this  case,  the  linear  transformation  Y  ==  SOX  +  50  was  used  to  put 
the  values  on  a  scale  between  0  and  100.  Scaling  results  are  shown  graphically  in  Figure 


MOE  Rank 


Figure  7.  Scale  Values  for  MOEs  in  First  Doctrinal  Bin 


Figures  8,  9,  10,  and  1 1  graphically  display  the  scaling  results  for  the  final  four  pages 
of  the  questionnaire. 
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Meo«ireorJEfrfccti<f«atess  (16Umntki 'I  >-  s'-C<^  L.Ease'ofUse  -Kted'orinmicq 


-  -  MeASttre«fEITecthmie^;;-(17M^  Cost  ErtseortJsc  Vt^iomancc 

.  Number  of  options  remaining:  equals  decision  points  open)X(#  options  for  each  decision  point) 

.  Time  to  decision  ratio: 

(time  lisue  order  to  echelon  -  lime  receipt  mis:ion  | 
time  ordered  to  Hart  execution  -  time  receipt  mission  J 
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time  ixsue  order  to  echelort  -  time  receipt  mission 
time  ordered  to  start  execution  -  time  receipt  mission 
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